3751-3800 of 10000 results (130ms)
2024-03-19 §
15:02 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: mariadb::misc::phabricator [production]
15:00 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
15:00 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
15:00 <effie> restart kartotherian on eqiad [production]
14:58 <effie> pooling kartotherian on codfw back [production]
14:57 <jiji@cumin1002> conftool action : set/pooled=true; selector: dnsdisc=kartotherian,name=codfw [production]
14:57 <jiji@cumin1002> conftool action : set/pooled=true; selector: dnsdisc=kartotherian,name=eqiad [production]
14:55 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-role for role: mariadb::misc::phabricator [production]
14:42 <effie> Traffic+Services switchover complete, codfw is depooled - Τ357547 [production]
14:40 <jiji@cumin1002> END (PASS) - Cookbook sre.discovery.datacenter (exit_code=0) depool all services in codfw: Northward DC Switchover, March 2024 - T357547 [production]
14:22 <effie> depooling services from codfw - T357547 [production]
14:16 <jiji@cumin1002> START - Cookbook sre.discovery.datacenter depool all services in codfw: Northward DC Switchover, March 2024 - T357547 [production]
14:07 <effie> Completely depool codfw from user traffic - T357547 [production]
13:55 <herron> kafka-logging1001:~# kafka reassign-partitions -reassignment-json-file rsyslog-info.json --execute --throttle 50000000 T326419 [production]
13:50 <herron> kafka-logging1001:~# kafka reassign-partitions -reassignment-json-file mediawiki.httpd.accesslog-sampled.json --execute --throttle 50000000 T326419 [production]
13:45 <cgoubert@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop: apply [production]
13:44 <cgoubert@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop: apply [production]
13:43 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop: apply [production]
13:43 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop: apply [production]
13:42 <cgoubert@deploy2002> helmfile [staging] DONE helmfile.d/services/changeprop: apply [production]
13:42 <cgoubert@deploy2002> helmfile [staging] START helmfile.d/services/changeprop: apply [production]
13:42 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop: apply [production]
13:41 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop: apply [production]
13:41 <claime> Deploying changeprop and changeprop-jobqueue - T353876 [production]
13:38 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop: apply [production]
13:38 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop: apply [production]
13:19 <hashar> Restarting CI Jenkins [production]
13:08 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on restbase1024.eqiad.wmnet with reason: Decommissioning — T354561 [production]
13:08 <eevans@cumin1002> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on restbase1024.eqiad.wmnet with reason: Decommissioning — T354561 [production]
13:07 <cgoubert@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
13:07 <cgoubert@deploy2002> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]
13:07 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply [production]
13:07 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply [production]
13:06 <claime> manually adding 20 replicas to mw-parsoid to help with big reparse [production]
12:59 <cmooney@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox [production]
12:59 <cmooney@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox [production]
12:59 <cmooney@cumin1002> END (FAIL) - Cookbook sre.netbox.update-extras (exit_code=1) rolling restart_daemons on A:netbox-canary [production]
12:49 <cgoubert@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
12:49 <cmooney@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary [production]
12:48 <cgoubert@deploy2002> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]
12:48 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply [production]
12:48 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply [production]
12:11 <slyngs> Switch idp-test to upgraded Bookworm host [production]
11:45 <cgoubert@deploy2002> Finished scap: mediawiki: Route /w/CREDITS and /w/COPYING to /w/static.php - gerrit:1012627 - T359643 (duration: 06m 59s) [production]
11:40 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
11:39 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
11:21 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
11:21 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
11:19 <oblivian@deploy2002> dancy and oblivian: Continuing with sync [production]
11:18 <oblivian@deploy2002> dancy and oblivian: Backport for [[gerrit:1012427|static.php: Handle COPYING and CREDITS files (T359643)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]