container-builder issueshttps://code.ornl.gov/olcf/container-builder/-/issues2018-03-01T12:49:16Zhttps://code.ornl.gov/olcf/container-builder/-/issues/38Fix deployment2018-03-01T12:49:16ZSimpson, Adam BFix deploymentDeployment should stage to a test area, run tests, and then replace the production instance.Deployment should stage to a test area, run tests, and then replace the production instance.https://code.ornl.gov/olcf/container-builder/-/issues/37animated ellipses don't work2018-03-01T12:48:07ZSimpson, Adam Banimated ellipses don't workA single dot is displayedA single dot is displayedhttps://code.ornl.gov/olcf/container-builder/-/issues/36blarg2018-03-01T12:48:00ZSimpson, Adam BblargJan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error creating builder, retrying in five seconds: 1
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Checking reserve builder count with 0 reserve builders
Jan 29 20:32:02...Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error creating builder, retrying in five seconds: 1
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Checking reserve builder count with 0 reserve builders
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Parsing builder output
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error parsing JSON builder output: <unspecified file>(1): expected value
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] builder request returned non zero: 1
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error creating builder, retrying in five seconds: 4
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Checking reserve builder count with 0 reserve builders
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Parsing builder output
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error parsing JSON builder output: <unspecified file>(1): expected value
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] builder request returned non zero: 1
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error creating builder, retrying in five seconds: 2
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Checking reserve builder count with 0 reserve builders
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Parsing builder output
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error parsing JSON builder output: <unspecified file>(1): expected value
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] builder request returned non zero: 1
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error creating builder, retrying in five seconds: 3
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Checking reserve builder count with 0 reserve builders
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Parsing builder output
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error parsing JSON builder output: <unspecified file>(1): expected value
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] builder request returned non zero: 1
Jan 29 20:32:02 builderqueue builder-queue[1037]: [ERROR] Error creating builder, retrying in five seconds: 0
Jan 29 20:32:02 builderqueue builder-queue[1037]: [INFO] Checking reserve builder count with 0 reserve buildershttps://code.ornl.gov/olcf/container-builder/-/issues/35retry create builder if it fails2018-01-06T22:56:35ZSimpson, Adam Bretry create builder if it failsIf a builder fails to be created we need to retry itIf a builder fails to be created we need to retry ithttps://code.ornl.gov/olcf/container-builder/-/issues/34Don't use waiting animation when debug output enabled2018-01-29T22:21:05ZSimpson, Adam BDon't use waiting animation when debug output enabledDebug output may print during animation making a mess of stderr/outDebug output may print during animation making a mess of stderr/outhttps://code.ornl.gov/olcf/container-builder/-/issues/33Make better use of exceptions2018-01-06T21:19:42ZSimpson, Adam BMake better use of exceptionshttps://code.ornl.gov/olcf/container-builder/-/issues/32Severity level doesn't show up in logs2017-12-29T13:01:18ZSimpson, Adam BSeverity level doesn't show up in logs2017-12-29 05:11:32.679480 []: Failed to fetch server list2017-12-29 05:11:32.679480 []: Failed to fetch server listhttps://code.ornl.gov/olcf/container-builder/-/issues/31Client should exit immediately on failure2017-12-29T13:34:09ZSimpson, Adam BClient should exit immediately on failure```
[atj@titan-ext7]$ container_builder --arch=ppc64le test.img test.def
2017-Dec-17 22:42:39 [SUCCESS] Connecting to BuilderQueue:
2017-Dec-17 22:42:40 [SUCCESS] Requesting Builder:
2017-Dec-17 22:42:40 [SUCCESS] Connecting to Builder...```
[atj@titan-ext7]$ container_builder --arch=ppc64le test.img test.def
2017-Dec-17 22:42:39 [SUCCESS] Connecting to BuilderQueue:
2017-Dec-17 22:42:40 [SUCCESS] Requesting Builder:
2017-Dec-17 22:42:40 [SUCCESS] Connecting to Builder:
2017-Dec-17 22:42:40 [INFO] Sending definition: test.def
0% 10 20 30 40 50 60 70 80 90 100%
|----|----|----|----|----|----|----|----|----|----|
***************************************************
2017-Dec-17 22:42:40 [INFO] Start of Singularity builder output:
ERROR: Unknown container build Singularity recipe format: ./container.def
ABORT: Aborting with RETVAL=255
Cleaning up...
2017-Dec-17 22:42:41 [INFO] Sending finished container: test.img
2017-Dec-17 22:42:41 [INFO] Error receiving headerEnd of file
2017-Dec-17 22:42:41 [INFO] Recieved message hader not of file type
2017-Dec-17 22:42:41 [SUCCESS] Container received: test.img
```https://code.ornl.gov/olcf/container-builder/-/issues/30Fix whatever caused the queue to go down...2017-12-13T00:44:09ZSimpson, Adam BFix whatever caused the queue to go down...```
cades@builderqueue:~$ tail -f /home/queue/ContainerBuilder.log
2017-12-11 17:22:31.440361 (656) [128.219.164.233:43774] : Established connection
2017-12-11 17:22:31.542644 (657) [128.219.164.233:43774] : Connection initial request...```
cades@builderqueue:~$ tail -f /home/queue/ContainerBuilder.log
2017-12-11 17:22:31.440361 (656) [128.219.164.233:43774] : Established connection
2017-12-11 17:22:31.542644 (657) [128.219.164.233:43774] : Connection initial request error received bad message type: Bad message
2017-12-11 17:22:31.542674 (658) [128.219.164.233:43774] : Ending connection
2017-12-11 17:22:31.644552 (659) [128.219.164.233:43850] : Established connection
2017-12-11 17:22:31.745598 (660) [128.219.164.233:43850] : Connection initial request error received bad message type: Bad message
2017-12-11 17:22:31.746069 (661) [128.219.164.233:43850] : Ending connection
2017-12-11 17:22:31.847465 (662) [128.219.164.233:43934] : Established connection
2017-12-11 17:22:32.954434 (663) Running command: /home/queue/GetBuilders
2017-12-11 17:22:35.700940 (664) Running command: /home/queue/GetBuilders
2017-12-11 17:22:38.649908 (665) Running command: /home/queue/GetBuilders
```
```
cades@builderqueue:~$ sudo systemctl status BuilderQueue
● BuilderQueue.service - BuilderQueue daemon
Loaded: loaded (/etc/systemd/system/BuilderQueue.service; enabled; vendor preset: enabled)
Active: failed (Result: core-dump) since Mon 2017-12-11 17:22:39 UTC; 20h ago
Main PID: 964 (code=dumped, signal=ABRT)
Dec 11 16:53:51 builderqueue systemd[1]: Started BuilderQueue daemon.
Dec 11 17:22:39 builderqueue BuilderQueue[964]: terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
Dec 11 17:22:39 builderqueue BuilderQueue[964]: what(): remote_endpoint: Transport endpoint is not connected
Dec 11 17:22:39 builderqueue systemd[1]: BuilderQueue.service: Main process exited, code=dumped, status=6/ABRT
Dec 11 17:22:39 builderqueue systemd[1]: BuilderQueue.service: Unit entered failed state.
Dec 11 17:22:39 builderqueue systemd[1]: BuilderQueue.service: Failed with result 'core-dump'.
```https://code.ornl.gov/olcf/container-builder/-/issues/29Have timeout of client connecting to builder2017-12-10T14:16:52ZSimpson, Adam BHave timeout of client connecting to builderhttps://code.ornl.gov/olcf/container-builder/-/issues/28Accept a uri for the definition2018-01-29T22:20:52ZSimpson, Adam BAccept a uri for the definition*Created by: AdamSimpson*
Allow more generic URI instead of just a local file*Created by: AdamSimpson*
Allow more generic URI instead of just a local filehttps://code.ornl.gov/olcf/container-builder/-/issues/27Use SSL2017-12-20T17:25:18ZSimpson, Adam BUse SSL*Created by: AdamSimpson*
*Created by: AdamSimpson*
https://code.ornl.gov/olcf/container-builder/-/issues/26Build ppc64le containers with qemu2018-01-06T21:19:53ZSimpson, Adam BBuild ppc64le containers with qemu*Created by: AdamSimpson*
*Created by: AdamSimpson*
https://code.ornl.gov/olcf/container-builder/-/issues/25Use client to test if we're in a tty and inform the builder2017-12-29T13:01:39ZSimpson, Adam BUse client to test if we're in a tty and inform the builder*Created by: AdamSimpson*
Don't run through `unbuffer` if the client isn't running a tty*Created by: AdamSimpson*
Don't run through `unbuffer` if the client isn't running a ttyhttps://code.ornl.gov/olcf/container-builder/-/issues/24Make sure OpenStack server delete actually deletes the server2017-12-20T17:25:18ZSimpson, Adam BMake sure OpenStack server delete actually deletes the server*Created by: AdamSimpson*
I have seen an "ERROR" state...*Created by: AdamSimpson*
I have seen an "ERROR" state...https://code.ornl.gov/olcf/container-builder/-/issues/23Attempt to make OpenStack the authoritative source of server information2018-01-06T21:20:01ZSimpson, Adam BAttempt to make OpenStack the authoritative source of server information*Created by: AdamSimpson*
Reduce the possibility of the queue's view of what servers are up/down coming out of sync with OpenStacks view*Created by: AdamSimpson*
Reduce the possibility of the queue's view of what servers are up/down coming out of sync with OpenStacks viewhttps://code.ornl.gov/olcf/container-builder/-/issues/22install yum on builder2017-12-20T17:25:18ZSimpson, Adam Binstall yum on builder*Created by: AdamSimpson*
*Created by: AdamSimpson*
https://code.ornl.gov/olcf/container-builder/-/issues/21checksum files on transfer2017-12-20T17:25:18ZSimpson, Adam Bchecksum files on transfer*Created by: AdamSimpson*
Add checksum to header*Created by: AdamSimpson*
Add checksum to headerhttps://code.ornl.gov/olcf/container-builder/-/issues/20Make file chunk size default to the asio buffer sizes2017-12-20T17:25:18ZSimpson, Adam BMake file chunk size default to the asio buffer sizes*Created by: AdamSimpson*
*Created by: AdamSimpson*
https://code.ornl.gov/olcf/container-builder/-/issues/19QUEUE_HOST / QUEUE_PORT give cryptic errors when undefined2017-12-20T17:25:18ZFrench, RobertQUEUE_HOST / QUEUE_PORT give cryptic errors when undefinedTrying to connect be like
```
~/P/C/build ❯❯❯ ./ContainerBuilderClient ../ContainerBuilderTitan.def poop.img
Attempting to connect to BuilderQueue: Failed to build container: QUEUE_HOST: Operation not supported
```
same for QUE...Trying to connect be like
```
~/P/C/build ❯❯❯ ./ContainerBuilderClient ../ContainerBuilderTitan.def poop.img
Attempting to connect to BuilderQueue: Failed to build container: QUEUE_HOST: Operation not supported
```
same for QUEUE_PORT. The client should complain in a more straight-forward way if they are missing (though I reckon this will be deployed via environment modules that would set these variables anyhow?)