9651-9700 of 10000 results (77ms)
2019-09-30 ยง
19:01 <hashar> Reloading Zuul for https://gerrit.wikimedia.org/r/539630 [releng]
18:53 <hashar> Create jobs wmf-quibble-selenium-hhvm-docker wmf-quibble-selenium-php72-docker | https://gerrit.wikimedia.org/r/#/c/integration/config/+/539630/ [releng]
17:36 <twentyafterfour@deploy1001> Finished deploy [releng/phatality@62e2870]: fix T234223 (duration: 03m 03s) [production]
17:33 <twentyafterfour@deploy1001> Started deploy [releng/phatality@62e2870]: fix T234223 [production]
17:22 <twentyafterfour@deploy1001> Started deploy [releng/phatality@62e2870]: fix T234223 [production]
17:18 <twentyafterfour@deploy1001> Finished deploy [releng/phatality@62e2870]: fix T234223 (duration: 00m 05s) [production]
17:18 <twentyafterfour@deploy1001> Started deploy [releng/phatality@62e2870]: fix T234223 [production]
17:15 <bstorm_> restart wikilabels postgresql [clouddb-services]
17:15 <twentyafterfour@deploy1001> deploy aborted: fix T234223 (duration: 06m 24s) [production]
17:15 <bstorm_> restart osmdb on clouddb1003 [clouddb-services]
17:10 <twentyafterfour> deploy failed [production]
17:09 <twentyafterfour@deploy1001> Started deploy [releng/phatality@62e2870]: fix T234223 [production]
17:08 <twentyafterfour> deploying minor update to phatality to fix T234223 [production]
16:35 <cdanis@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
16:34 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 0aa4b4b5ab9a2e4 (duration: 00m 57s) [production]
16:34 <ppchelko@deploy1001> Finished deploy [cpjobqueue/deploy@79db711]: Take job domain into account for deduplication T234226 (duration: 01m 17s) [production]
16:32 <krinkle@deploy1001> Synchronized wmf-config/abusefilter.php: 0aa4b4b5ab9a2e4 (duration: 00m 57s) [production]
16:32 <ppchelko@deploy1001> Started deploy [cpjobqueue/deploy@79db711]: Take job domain into account for deduplication T234226 [production]
16:25 <cdanis@cumin1001> START - Cookbook sre.ganeti.makevm [production]
16:25 <cdanis@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) [production]
16:25 <cdanis@cumin1001> START - Cookbook sre.ganeti.makevm [production]
16:14 <Amir1> soft-rebooting codesearch4 [codesearch]
15:49 <Amir1> ladsgroup@codesearch4:~$ sudo MODE=restart /srv/codesearch/manage.sh (T234211) [codesearch]
15:49 <moritzm> installing console-setup bugfixes from Buster 10.1 point release [production]
15:46 <cdanis@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) [production]
15:46 <cdanis@cumin1001> START - Cookbook sre.ganeti.makevm [production]
15:42 <moritzm> failover Ganeti master in codfw to ganeti2001 [production]
15:23 <James_F> Zuul: Added TheSandDoctor to the CI whitelist. [releng]
15:09 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:09 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
14:59 <tgr> restarted docker on discuss-space for T234218 [discourse]
14:44 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:44 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
14:29 <moritzm> draining ganeti2007 for upcoming reboot (combined kernel/qemu security updates) [production]
14:17 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:17 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
14:08 <moritzm> draining ganeti2006 for upcoming reboot (combined kernel/qemu security updates) [production]
14:00 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:00 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
13:54 <moritzm> draining ganeti2005 for upcoming reboot (combined kernel/qemu security updates) [production]
13:49 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:49 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
13:10 <hashar> integration-castor03 : sudo mkdir -p /usr/local/lib/python2.7/dist-packages [releng]
12:53 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:51 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:49 <hashar> Fixed cumin on integration project # T234203 [releng]
12:33 <kart_> Update cxserver to 2019-09-26-034732-production (T233834, T232674, T233085) [production]
12:29 <@> helmfile [EQIAD] Ran 'apply' command on namespace 'cxserver' for release 'production' . [production]
12:29 <jbond42> offline puppetmaster2002 to reimage https://gerrit.wikimedia.org/r/c/operations/puppet/+/539322 [production]
12:27 <@> helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . [production]