3201-3250 of 10000 results (72ms)
2019-10-28 §
10:39 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:546442| Bumping portals to master (T128546)]] (duration: 00m 54s) [production]
08:48 <godog> bump udp_localhost kafka-logging topics to 6 partitions and roll-restart logstash and rsyslog - T215904 [production]
08:26 <volans> manually cleanup changes reverted in https://gerrit.wikimedia.org/r/546407 on icinga[12]001 - T222074 [production]
08:25 <moritzm> installing file/libmagic security updates [production]
08:16 <mobrovac@deploy1001> Finished deploy [restbase/deploy@447981b]: Parsoid: Shim content-language and vary headers only for the PHP variant - T230791 (duration: 13m 42s) [production]
08:15 <godog> swift eqiad-prod: final weight to ms-be105[1-6] - T232367 [production]
08:02 <mobrovac@deploy1001> Started deploy [restbase/deploy@447981b]: Parsoid: Shim content-language and vary headers only for the PHP variant - T230791 [production]
07:40 <mobrovac@deploy1001> Finished deploy [restbase/deploy@c500d7a]: Add the Parsoid proxy for JS/PHP variants, add top mediarequests end point and add mnwwiki and ge.wm.org - T230791 T235744 T236389 (duration: 13m 44s) [production]
07:40 <elukey@deploy1001> Finished deploy [eventlogging/analytics@0f1ad6d]: Move codebase to Python3 - second attempt (duration: 00m 05s) [production]
07:40 <elukey@deploy1001> Started deploy [eventlogging/analytics@0f1ad6d]: Move codebase to Python3 - second attempt [production]
07:37 <elukey> upload archiva 2.2.4-1 to wikimedia-stretch (fix to avoid overriding archiva.xml upon install) [production]
07:27 <mobrovac@deploy1001> Started deploy [restbase/deploy@c500d7a]: Add the Parsoid proxy for JS/PHP variants, add top mediarequests end point and add mnwwiki and ge.wm.org - T230791 T235744 T236389 [production]
07:25 <mobrovac@deploy1001> Finished deploy [restbase/deploy@c500d7a] (dev-cluster): Add the Parsoid proxy for JS/PHP variants, add top mediarequests end point and add mnwwiki and ge.wm.org (duration: 02m 37s) [production]
07:22 <mobrovac@deploy1001> Started deploy [restbase/deploy@c500d7a] (dev-cluster): Add the Parsoid proxy for JS/PHP variants, add top mediarequests end point and add mnwwiki and ge.wm.org [production]
2019-10-26 §
11:30 <XioNoX> restart cr3-esams [production]
11:01 <XioNoX> re0.cr3-esams> request chassis routing-engine master switch [production]
2019-10-25 §
22:54 <mutante> moscovium rm /dev/shm/envoy_shared_memory_0 to revive envoy which failed to run after changing ports and reinstalling it (T180641) [production]
22:42 <mutante> moscovium - manually deleting envoy listener on 1443 and letting puppet recreate config because it's not removed if you change the port (T180641) [production]
21:55 <mutante> running puppet on ulsfo cp-ats servers to pick up config change for RT backend [production]
20:42 <twentyafterfour@deploy1001> Finished deploy [design/style-guide@c69242e]: deploying design/style-guide for demonstration purposes (duration: 00m 06s) [production]
20:41 <twentyafterfour@deploy1001> Started deploy [design/style-guide@c69242e]: deploying design/style-guide for demonstration purposes [production]
20:04 <twentyafterfour@deploy1001> Finished deploy [design/style-guide@c69242e]: test deploy design/style-guide (duration: 00m 10s) [production]
20:04 <twentyafterfour@deploy1001> Started deploy [design/style-guide@c69242e]: test deploy design/style-guide [production]
17:49 <bblack@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:47 <bblack@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:26 <bblack> lvs3005 - reimaging to fix partman issue, high-traffic1 (text) to lvs3007 for the duration [production]
16:43 <bblack@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:40 <bblack@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:19 <bblack> lvs3006 - reimaging to fix partman issue, high-traffic2 (upload/maps) to lvs3007 for the duration [production]
16:19 <crusnov@deploy1001> Finished deploy [netbox/deploy@0f4c92d]: deploy netbox scripts update (netbox1001) T223292 (duration: 13m 31s) [production]
16:05 <crusnov@deploy1001> Started deploy [netbox/deploy@0f4c92d]: deploy netbox scripts update (netbox1001) T223292 [production]
16:04 <crusnov@deploy1001> Finished deploy [netbox/deploy@0f4c92d]: deploy netbox scripts update (netbox2001) T223292 (duration: 00m 43s) [production]
16:04 <crusnov@deploy1001> Started deploy [netbox/deploy@0f4c92d]: deploy netbox scripts update (netbox2001) T223292 [production]
15:35 <robh> ps1-oe14-esams ip info set, rebooting (wont affect servers) via T184066 [production]
15:03 <gehel@cumin1001> END (ERROR) - Cookbook sre.wdqs.data-reload (exit_code=97) [production]
15:01 <bblack@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
15:00 <bblack@cumin1001> START - Cookbook sre.hosts.decommission [production]
14:41 <bblack> cr[23]-esams: re-route ns2 IP to ganeti3003 [production]
14:36 <gehel@cumin1001> START - Cookbook sre.wdqs.data-reload [production]
14:32 <crusnov@deploy1001> Finished deploy [netbox/deploy@690f9ae]: deploy netbox scripts (netbox2001) -T223292 (duration: 00m 44s) [production]
14:31 <crusnov@deploy1001> Started deploy [netbox/deploy@690f9ae]: deploy netbox scripts (netbox2001) -T223292 [production]
14:30 <crusnov@deploy1001> Finished deploy [netbox/deploy@690f9ae]: deploy netbox scripts (netbox2001) T223292 (duration: 00m 05s) [production]
14:30 <crusnov@deploy1001> Started deploy [netbox/deploy@690f9ae]: deploy netbox scripts (netbox2001) T223292 [production]
14:28 <crusnov@deploy1001> Finished deploy [netbox/deploy@690f9ae]: deploy netbox scripts T223292 (duration: 01m 02s) [production]
14:27 <crusnov@deploy1001> Started deploy [netbox/deploy@690f9ae]: deploy netbox scripts T223292 [production]
14:17 <gehel@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) [production]
14:15 <gehel@cumin1001> START - Cookbook sre.wdqs.data-reload [production]
14:10 <gehel@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) [production]
14:10 <gehel@cumin1001> START - Cookbook sre.wdqs.data-reload [production]
14:09 <bblack> reboot ganeti3003 [production]