5651-5700 of 10000 results (95ms)
2020-02-11 §
07:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2086:3318 T239453', diff saved to https://phabricator.wikimedia.org/P10373 and previous config saved to /var/cache/conftool/dbconfig/20200211-071936-marostegui.json [production]
07:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2085:3318 T239453', diff saved to https://phabricator.wikimedia.org/P10372 and previous config saved to /var/cache/conftool/dbconfig/20200211-071639-marostegui.json [production]
07:07 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1107 for 10.4 testing - T242702', diff saved to https://phabricator.wikimedia.org/P10371 and previous config saved to /var/cache/conftool/dbconfig/20200211-070720-marostegui.json [production]
07:01 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
06:59 <marostegui> Stop haproxy on dbproxy1001 - T244463 [production]
06:59 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
06:58 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
06:57 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
06:48 <marostegui> Remove grants in m1 for dbproxy1001 - T231280 [production]
06:25 <vgutierrez> depool cp5001 & cp5008 and reimage as buster - T242093 [production]
06:18 <marostegui> Failover m1-master from dbproxy1014 to dbproxy1012 - T202367 [production]
00:26 <ebernhardson@deploy1001> Synchronized php-1.35.0-wmf.18/skins/MinervaNeue: SWAT: Revert: Reduce userContributions icon code (duration: 01m 06s) [production]
00:20 <ebernhardson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: Give NS_HELP same weight as NS_MAIN in search on wikitech (duration: 01m 06s) [production]
00:15 <ebernhardson@deploy1001> Synchronized wmf-config/: SWAT: Enable SpecialMute page on all wikis (duration: 01m 06s) [production]
2020-02-10 §
23:54 <bd808> Shutdown webservice and removed symling from $HOME/public_html to /home/myst/public_html; contained TOU violating adminer software (phpmyadmin clone) [tools.coord]
23:42 <James_F> Zuul: [parsoid/deploy] Make PHP tests voting T239642 [releng]
23:39 <bstorm_> updated tools-manifest to 0.21 on aptly for stretch [tools]
23:30 <robh> cp108[23] returned to service via T243167 [production]
23:28 <legoktm> restarting zuul [production]
23:28 <legoktm> restarting zuul [releng]
23:26 <reedy@deploy1001> Synchronized php-1.35.0-wmf.18/extensions/OATHAuth/src/Key/TOTPKey.php: T244308 (duration: 01m 04s) [production]
23:25 <reedy@deploy1001> Synchronized php-1.35.0-wmf.16/extensions/OATHAuth/src/Key/TOTPKey.php: T244308 (duration: 01m 07s) [production]
23:06 <robh> cp108[01] returned to service, cp108[23] offline for bios update via T243167 [production]
22:51 <bstorm_> all docker images now use webservice 0.62 [tools]
22:50 <chasemp> phab1001:~# sudo /srv/phab/phabricator/bin/bulk make-silent --id 2164 [production]
22:45 <sbassett@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Add authevents as monolog channel (duration: 01m 06s) [production]
22:43 <robh> cp107[789] returned to service, cp108[01] offline for bios update via T243167 [production]
22:42 <robh> cp107[89] returned to service, cp108[01] offline for bios update via T243167 [production]
22:24 <James_F> Zuul: Add tox for operations/homer/mock-private T244690 [releng]
22:01 <bd808> Manually starting webservices for tools that were running on tools-sgewebgrid-lighttpd-0928 (T244791) [tools]
21:58 <robh> cp107[56] returned to service, cp107[78] offline for bios update via T243167 [production]
21:47 <bd808> Depooling tools-sgewebgrid-lighttpd-0928 (T244791) [tools]
21:43 <arlolra> Updated Parsoid to 612106d2 (T244412, T244413, T242746, T235273, T235307, T238845, T204618, T240054) [production]
21:38 <robh> cp1075 & cp1076 offline for bios updates per T243167 [production]
21:36 <robh> cp1075 and cp1076 going offline for bios updates. This will cause a bit of cp irc icinga noise, but no paging. Not putting into maint mode, as there is no way to maint mode the noisest check (which checks all backends and thus shouldnt be disabled) [production]
21:33 <arlolra@deploy1001> Finished deploy [parsoid/deploy@d2d4870]: Updating Parsoid to 612106d2 (duration: 10m 26s) [production]
21:32 <XioNoX> clamp tcp-mss on cr2-eqiad:xe-3/3/3 [production]
21:25 <bstorm_> upgraded toollabs-webservice package for tools to 0.62 T244293 T244289 T234617 T156626 [tools]
21:23 <arlolra@deploy1001> Started deploy [parsoid/deploy@d2d4870]: Updating Parsoid to 612106d2 [production]
21:18 <bstorm_> upgraded toollabs-webservice package for stretch toolsbeta to 0.62 T244293 T244289 T234617 T156626 [toolsbeta]
21:12 <halfak@deploy1001> Finished deploy [ores/deploy@a6f4f14]: T242705 (duration: 12m 18s) [production]
21:00 <halfak@deploy1001> Started deploy [ores/deploy@a6f4f14]: T242705 [production]
20:55 <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.16/extensions/MachineVision: MachineVision: Fix page id parsing from imageinfo results (T244752) (duration: 01m 11s) [production]
20:14 <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.18/extensions/MachineVision: MachineVision: Fix page id parsing from imageinfo results (T244752) (duration: 01m 15s) [production]
19:48 <James_F> Add mahveotm to CI whitelist [releng]
19:31 <ppchelko@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:570393]] Config: Session Store: Switch group0 and group1 to kask-session T243106 (duration: 01m 06s) [production]
19:28 <mutante> Gerrit - added eevans to 'wmf-deployment' group (T244508) [production]
19:12 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: T242122 Load new EventStreamConfig extension if so configured (duration: 01m 06s) [production]
19:07 <jforrester@deploy1001> Scap failed!: Call to mwscript eval.php stderr: not empty [production]
19:06 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T242122 Set default of wmgUseEventStreamConfig false everywhere (duration: 01m 06s) [production]