4901-4950 of 10000 results (104ms)
2024-05-15 §
07:04 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
07:04 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
00:29 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1181 (T352010)', diff saved to https://phabricator.wikimedia.org/P62399 and previous config saved to /var/cache/conftool/dbconfig/20240515-002923-ladsgroup.json [production]
00:29 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1181.eqiad.wmnet with reason: Maintenance [production]
00:29 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1181.eqiad.wmnet with reason: Maintenance [production]
00:29 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1174 (T352010)', diff saved to https://phabricator.wikimedia.org/P62398 and previous config saved to /var/cache/conftool/dbconfig/20240515-002900-ladsgroup.json [production]
00:13 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1174', diff saved to https://phabricator.wikimedia.org/P62397 and previous config saved to /var/cache/conftool/dbconfig/20240515-001352-ladsgroup.json [production]
2024-05-14 §
23:58 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1174', diff saved to https://phabricator.wikimedia.org/P62396 and previous config saved to /var/cache/conftool/dbconfig/20240514-235844-ladsgroup.json [production]
23:43 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1174 (T352010)', diff saved to https://phabricator.wikimedia.org/P62395 and previous config saved to /var/cache/conftool/dbconfig/20240514-234337-ladsgroup.json [production]
22:48 <zabe> start running migrateGuSalt.php in screen session # T364435 [production]
22:22 <zabe> zabe@mwmaint1002:/tmp/upload$ mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user="Yann" . # T364877 [production]
22:06 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2162 (T364299)', diff saved to https://phabricator.wikimedia.org/P62394 and previous config saved to /var/cache/conftool/dbconfig/20240514-220640-marostegui.json [production]
22:06 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2162.codfw.wmnet with reason: Maintenance [production]
22:06 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2162.codfw.wmnet with reason: Maintenance [production]
22:06 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T364299)', diff saved to https://phabricator.wikimedia.org/P62393 and previous config saved to /var/cache/conftool/dbconfig/20240514-220617-marostegui.json [production]
21:51 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P62392 and previous config saved to /var/cache/conftool/dbconfig/20240514-215109-marostegui.json [production]
21:39 <eileen> civicrm upgraded from c7b0dfbb to 9268acf3 [production]
21:36 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P62391 and previous config saved to /var/cache/conftool/dbconfig/20240514-213601-marostegui.json [production]
21:20 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T364299)', diff saved to https://phabricator.wikimedia.org/P62390 and previous config saved to /var/cache/conftool/dbconfig/20240514-212052-marostegui.json [production]
21:00 <cjming@deploy1002> Finished scap: Backport for [[gerrit:1031466|Override VE overlays in night-mode (T363861)]] (duration: 18m 44s) [production]
20:49 <vriley@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host kafka-main1009.mgmt.eqiad.wmnet with reboot policy FORCED [production]
20:48 <cjming@deploy1002> cjming and jdlrobson: Continuing with sync [production]
20:44 <dcausse@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
20:44 <cjming@deploy1002> cjming and jdlrobson: Backport for [[gerrit:1031466|Override VE overlays in night-mode (T363861)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:44 <dcausse@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
20:42 <cjming@deploy1002> Started scap: Backport for [[gerrit:1031466|Override VE overlays in night-mode (T363861)]] [production]
20:41 <cjming@deploy1002> Finished scap: Backport for [[gerrit:1031029|cirrus: Shift 25% of public wikis writes in eqiad to replacement updater (T363475)]] (duration: 15m 02s) [production]
20:29 <cjming@deploy1002> cjming and ebernhardson: Continuing with sync [production]
20:28 <cjming@deploy1002> cjming and ebernhardson: Backport for [[gerrit:1031029|cirrus: Shift 25% of public wikis writes in eqiad to replacement updater (T363475)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:28 <ebernhardson@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
20:28 <ebernhardson@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
20:26 <cjming@deploy1002> Started scap: Backport for [[gerrit:1031029|cirrus: Shift 25% of public wikis writes in eqiad to replacement updater (T363475)]] [production]
20:24 <cjming@deploy1002> Finished scap: Backport for [[gerrit:1031495|Enable night mode on Vector on testwiki, disable on Special:Homepage (T357699 T363814)]] (duration: 18m 40s) [production]
20:14 <ebernhardson@deploy1002> Finished deploy [airflow-dags/search@ecf603d]: update discolytics to 0.18.0 (duration: 00m 27s) [production]
20:14 <ebernhardson@deploy1002> Started deploy [airflow-dags/search@ecf603d]: update discolytics to 0.18.0 [production]
20:11 <cjming@deploy1002> jdlrobson and cjming: Continuing with sync [production]
20:08 <cjming@deploy1002> jdlrobson and cjming: Backport for [[gerrit:1031495|Enable night mode on Vector on testwiki, disable on Special:Homepage (T357699 T363814)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:08 <cdanis@deploy1002> helmfile [eqiad] DONE helmfile.d/services/opentelemetry-collector: apply [production]
20:07 <cdanis@deploy1002> helmfile [eqiad] START helmfile.d/services/opentelemetry-collector: apply [production]
20:06 <cdanis@deploy1002> helmfile [staging] DONE helmfile.d/services/opentelemetry-collector: apply [production]
20:06 <vriley@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host kafka-main1010.mgmt.eqiad.wmnet with reboot policy FORCED [production]
20:06 <cdanis@deploy1002> helmfile [staging] START helmfile.d/services/opentelemetry-collector: apply [production]
20:05 <cjming@deploy1002> Started scap: Backport for [[gerrit:1031495|Enable night mode on Vector on testwiki, disable on Special:Homepage (T357699 T363814)]] [production]
20:04 <cdanis@deploy1002> helmfile [staging] DONE helmfile.d/services/opentelemetry-collector: apply [production]
20:04 <cdanis@deploy1002> helmfile [staging] START helmfile.d/services/opentelemetry-collector: apply [production]
20:01 <jclark@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
19:53 <cdanis@deploy1002> helmfile [eqiad] DONE helmfile.d/services/opentelemetry-collector: apply [production]
19:53 <cdanis@deploy1002> helmfile [eqiad] START helmfile.d/services/opentelemetry-collector: apply [production]
19:47 <vriley@cumin1002> START - Cookbook sre.hosts.provision for host kafka-main1009.mgmt.eqiad.wmnet with reboot policy FORCED [production]
19:47 <vriley@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host kafka-main1009.mgmt.eqiad.wmnet with reboot policy FORCED [production]