|
2025-11-25
§
|
| 10:52 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1230.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:51 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1216.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:02 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1207.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:54 |
<jnuche@deploy2002> |
rebuilt and synchronized wikiversions files: group0 to 1.46.0-wmf.4 refs T408274 |
[production] |
| 09:14 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1200.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:25 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1185.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:36 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1161.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:46 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1159.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:26 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2205.codfw.wmnet with reason: Maintenance |
[production] |
| 05:03 |
<mwpresync@deploy2002> |
Pruned MediaWiki: 1.46.0-wmf.1 (duration: 03m 53s) |
[production] |
| 04:58 |
<mwpresync@deploy2002> |
Finished scap sync-world: testwikis to 1.46.0-wmf.4 refs T408274 (duration: 55m 00s) |
[production] |
| 04:03 |
<mwpresync@deploy2002> |
Started scap sync-world: testwikis to 1.46.0-wmf.4 refs T408274 |
[production] |
| 01:01 |
<mwpresync@deploy2002> |
Finished scap build-images: Publishing wmf/next image (duration: 01m 14s) |
[production] |
| 01:00 |
<mwpresync@deploy2002> |
Started scap build-images: Publishing wmf/next image |
[production] |
|
2025-11-24
§
|
| 22:12 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2239.codfw.wmnet with reason: Maintenance |
[production] |
| 21:03 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2227.codfw.wmnet with reason: Maintenance |
[production] |
| 20:56 |
<swfrench-wmf> |
updated local PHP CLI installation on deploy2002 to 8.3 - T405955 |
[production] |
| 20:51 |
<swfrench-wmf> |
updated local PHP CLI installation on deploy1003 to 8.3 - T405955 |
[production] |
| 19:57 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2209.codfw.wmnet with reason: Maintenance |
[production] |
| 18:50 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2194.codfw.wmnet with reason: Maintenance |
[production] |
| 18:36 |
<swfrench-wmf> |
deleted EtcdReplicationDown silence. f75c71c9-62d3-449f-860a-9b5e4570717a - T405950 |
[production] |
| 18:34 |
<swfrench-wmf> |
begin restarts of eqiad-associated confds, navtiming, requestctl - T405950 |
[production] |
| 18:31 |
<swfrench-wmf> |
manually transferred etcd-mirror replication source back to conf1009 - T405950 |
[production] |
| 18:21 |
<swfrench-wmf> |
manually transferred etcd-mirror replication source to conf1008 - T405950 |
[production] |
| 18:16 |
<swfrench-wmf> |
silenced EtcdReplicationDown. f75c71c9-62d3-449f-860a-9b5e4570717a - T405950 |
[production] |
| 17:44 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2190.codfw.wmnet with reason: Maintenance |
[production] |
| 16:36 |
<btullis@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-master1004.eqiad.wmnet,an-redacteddb1001.eqiad.wmnet,an-test-coord1001.eqiad.wmnet with reason: Prepping for switch swap |
[production] |
| 16:34 |
<btullis@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on stat1011.eqiad.wmnet with reason: Prepping for switch swap |
[production] |
| 16:34 |
<btullis@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-test-master1002.eqiad.wmnet with reason: Prepping for switch swap |
[production] |
| 16:33 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2177.codfw.wmnet with reason: Maintenance |
[production] |
| 16:32 |
<btullis@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on dse-k8s-worker[1011,1013,1019].eqiad.wmnet with reason: Prepping for switch swap |
[production] |
| 16:28 |
<mwpresync@deploy2002> |
Finished scap build-images: Publishing wmf/next image (duration: 13m 52s) |
[production] |
| 16:14 |
<mwpresync@deploy2002> |
Started scap build-images: Publishing wmf/next image |
[production] |
| 15:27 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2156.codfw.wmnet with reason: Maintenance |
[production] |
| 14:22 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2149.codfw.wmnet with reason: Maintenance |
[production] |
| 13:43 |
<ladsgroup@deploy2002> |
sync-world failed: <CalledProcessError> Command '['sudo', '-u', 'mwbuilder', '-n', '--', '/usr/bin/scap', 'mwscript', '--no-local-config', '--directory', '/srv/mediawiki-staging', '--user', 'www-data', '--', 'mergeMessageFileList.php', '--wiki=aawiki', '--force-version', '1.46.0-wmf.3', '--list-file', '/srv/mediawiki-staging/wmf-config/extension-list', '--output', '/tmp/tmp.Seyz9S1dDd']' returned |
[production] |
| 13:42 |
<ladsgroup@deploy2002> |
sync-world failed: <CalledProcessError> Command '['sudo', '-u', 'mwbuilder', '-n', '--', '/usr/bin/scap', 'mwscript', '--no-local-config', '--directory', '/srv/mediawiki-staging', '--user', 'www-data', '--', 'mergeMessageFileList.php', '--wiki=aawiki', '--force-version', '1.46.0-wmf.3', '--list-file', '/srv/mediawiki-staging/wmf-config/extension-list', '--output', '/tmp/tmp.1aRzXHW4OP']' returned |
[production] |
| 12:47 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1223.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:23 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:17 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db1187 gradually with 4 steps - repool after schema change test |
[production] |
| 12:13 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1240.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:31 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.pool db1187 gradually with 4 steps - repool after schema change test |
[production] |
| 11:28 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1187.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:26 |
<marostegui@cumin1003> |
END (ERROR) - Cookbook sre.mysql.pool (exit_code=97) db1187 gradually with 4 steps - repool after schema change test |
[production] |
| 11:23 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.pool db1187 gradually with 4 steps - repool after schema change test |
[production] |
| 11:22 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1212.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:21 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1187.eqiad.wmnet with reason: Maintenance |
[production] |
| 10:32 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1198.eqiad.wmnet with reason: Maintenance |
[production] |
| 09:41 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1189.eqiad.wmnet with reason: Maintenance |
[production] |
| 08:50 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1175.eqiad.wmnet with reason: Maintenance |
[production] |