3301-3350 of 10000 results (70ms)
2019-10-28 §
13:07 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:05 <effie> enable puppet on mw2* servers, depool and repool to reload apache - T229792 [production]
13:01 <jynus> stop db1114 for testing [production]
12:30 <ema> depool cp5007 and reimage as text_ats T227432 [production]
12:22 <effie> depool mw2150 [production]
11:56 <twentyafterfour@deploy1001> Finished deploy [phabricator/deployment@e4e2b22]: testing deployment of phabricator to phab1001 (duration: 00m 05s) [production]
11:56 <twentyafterfour@deploy1001> Started deploy [phabricator/deployment@e4e2b22]: testing deployment of phabricator to phab1001 [production]
11:34 <Urbanecm> EU SWAT done [production]
11:33 <urbanecm@deploy1001> Synchronized php-1.35.0-wmf.3/extensions/VisualEditor/includes/ApiVisualEditorEdit.php: SWAT: 8caf681: Dont log missing ETags when creating a new page, thats normal (T233320) (duration: 00m 54s) [production]
11:33 <effie> Disable puppet on mw* for 545652 - T229792 [production]
11:12 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: dd2f06c: Add Translate channel for the Translate extension (T221119) (duration: 00m 53s) [production]
11:07 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: ff17666: Adjust wgUploadNavigationUrl for azwiki to point to commons UpWiz (T236307) (duration: 00m 53s) [production]
11:05 <urbanecm@deploy1001> Synchronized dblists/commonsuploads.dblist: SWAT: 7e26ef4: Revert "Restrict uploads on azwiki" (T236307) (duration: 00m 53s) [production]
11:02 <moritzm> installing OpenJDK security updates on elastic* [production]
10:40 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:546442| Bumping portals to master (T128546)]] (duration: 00m 53s) [production]
10:39 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:546442| Bumping portals to master (T128546)]] (duration: 00m 54s) [production]
08:48 <godog> bump udp_localhost kafka-logging topics to 6 partitions and roll-restart logstash and rsyslog - T215904 [production]
08:26 <volans> manually cleanup changes reverted in https://gerrit.wikimedia.org/r/546407 on icinga[12]001 - T222074 [production]
08:25 <moritzm> installing file/libmagic security updates [production]
08:16 <mobrovac@deploy1001> Finished deploy [restbase/deploy@447981b]: Parsoid: Shim content-language and vary headers only for the PHP variant - T230791 (duration: 13m 42s) [production]
08:15 <godog> swift eqiad-prod: final weight to ms-be105[1-6] - T232367 [production]
08:02 <mobrovac@deploy1001> Started deploy [restbase/deploy@447981b]: Parsoid: Shim content-language and vary headers only for the PHP variant - T230791 [production]
07:40 <mobrovac@deploy1001> Finished deploy [restbase/deploy@c500d7a]: Add the Parsoid proxy for JS/PHP variants, add top mediarequests end point and add mnwwiki and ge.wm.org - T230791 T235744 T236389 (duration: 13m 44s) [production]
07:40 <elukey@deploy1001> Finished deploy [eventlogging/analytics@0f1ad6d]: Move codebase to Python3 - second attempt (duration: 00m 05s) [production]
07:40 <elukey@deploy1001> Started deploy [eventlogging/analytics@0f1ad6d]: Move codebase to Python3 - second attempt [production]
07:37 <elukey> upload archiva 2.2.4-1 to wikimedia-stretch (fix to avoid overriding archiva.xml upon install) [production]
07:27 <mobrovac@deploy1001> Started deploy [restbase/deploy@c500d7a]: Add the Parsoid proxy for JS/PHP variants, add top mediarequests end point and add mnwwiki and ge.wm.org - T230791 T235744 T236389 [production]
07:25 <mobrovac@deploy1001> Finished deploy [restbase/deploy@c500d7a] (dev-cluster): Add the Parsoid proxy for JS/PHP variants, add top mediarequests end point and add mnwwiki and ge.wm.org (duration: 02m 37s) [production]
07:22 <mobrovac@deploy1001> Started deploy [restbase/deploy@c500d7a] (dev-cluster): Add the Parsoid proxy for JS/PHP variants, add top mediarequests end point and add mnwwiki and ge.wm.org [production]
2019-10-26 §
11:30 <XioNoX> restart cr3-esams [production]
11:01 <XioNoX> re0.cr3-esams> request chassis routing-engine master switch [production]
2019-10-25 §
22:54 <mutante> moscovium rm /dev/shm/envoy_shared_memory_0 to revive envoy which failed to run after changing ports and reinstalling it (T180641) [production]
22:42 <mutante> moscovium - manually deleting envoy listener on 1443 and letting puppet recreate config because it's not removed if you change the port (T180641) [production]
21:55 <mutante> running puppet on ulsfo cp-ats servers to pick up config change for RT backend [production]
20:42 <twentyafterfour@deploy1001> Finished deploy [design/style-guide@c69242e]: deploying design/style-guide for demonstration purposes (duration: 00m 06s) [production]
20:41 <twentyafterfour@deploy1001> Started deploy [design/style-guide@c69242e]: deploying design/style-guide for demonstration purposes [production]
20:04 <twentyafterfour@deploy1001> Finished deploy [design/style-guide@c69242e]: test deploy design/style-guide (duration: 00m 10s) [production]
20:04 <twentyafterfour@deploy1001> Started deploy [design/style-guide@c69242e]: test deploy design/style-guide [production]
17:49 <bblack@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:47 <bblack@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:26 <bblack> lvs3005 - reimaging to fix partman issue, high-traffic1 (text) to lvs3007 for the duration [production]
16:43 <bblack@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:40 <bblack@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:19 <bblack> lvs3006 - reimaging to fix partman issue, high-traffic2 (upload/maps) to lvs3007 for the duration [production]
16:19 <crusnov@deploy1001> Finished deploy [netbox/deploy@0f4c92d]: deploy netbox scripts update (netbox1001) T223292 (duration: 13m 31s) [production]
16:05 <crusnov@deploy1001> Started deploy [netbox/deploy@0f4c92d]: deploy netbox scripts update (netbox1001) T223292 [production]
16:04 <crusnov@deploy1001> Finished deploy [netbox/deploy@0f4c92d]: deploy netbox scripts update (netbox2001) T223292 (duration: 00m 43s) [production]
16:04 <crusnov@deploy1001> Started deploy [netbox/deploy@0f4c92d]: deploy netbox scripts update (netbox2001) T223292 [production]
15:35 <robh> ps1-oe14-esams ip info set, rebooting (wont affect servers) via T184066 [production]
15:03 <gehel@cumin1001> END (ERROR) - Cookbook sre.wdqs.data-reload (exit_code=97) [production]