4251-4300 of 10000 results (40ms)
2024-04-18 §
09:46 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1201.eqiad.wmnet with reason: Maintenance [production]
09:46 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1201.eqiad.wmnet with reason: Maintenance [production]
08:59 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1187.eqiad.wmnet with reason: Maintenance [production]
08:59 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1187.eqiad.wmnet with reason: Maintenance [production]
08:11 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1180.eqiad.wmnet with reason: Maintenance [production]
08:11 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1180.eqiad.wmnet with reason: Maintenance [production]
07:23 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
07:23 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
06:35 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1165.eqiad.wmnet with reason: Maintenance [production]
06:34 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1165.eqiad.wmnet with reason: Maintenance [production]
2024-04-17 §
21:50 <mutante> deploying scap config change (gerrit:1020321) - [cumin2002:~] $ sudo cumin -b 4 -s 40 'C:scap AND mw*' 'run-puppet-agent' T359643 [production]
18:35 <dancy@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.43.0-wmf.1 refs T361395 [production]
10:34 <akosiaris> apply the coredns patches for bumping instances from 4 to 6. They are noop, I am applying them to update helm's state. [production]
2024-04-16 §
21:16 <cjming@deploy1002> Finished scap: Backport for [[gerrit:1020292|[phase 4] Vector-2022.js should no longer load legacy Vector site and user scripts/styles (T301212)]] (duration: 18m 26s) [production]
21:01 <cjming@deploy1002> cjming and jdlrobson: Backport for [[gerrit:1020292|[phase 4] Vector-2022.js should no longer load legacy Vector site and user scripts/styles (T301212)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:57 <cjming@deploy1002> Started scap: Backport for [[gerrit:1020292|[phase 4] Vector-2022.js should no longer load legacy Vector site and user scripts/styles (T301212)]] [production]
19:08 <dancy@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.43.0-wmf.1 refs T361395 [production]
16:01 <sukhe@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on cp1115.eqiad.wmnet with reason: testing PXE boot issues [production]
16:00 <sukhe@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on cp1115.eqiad.wmnet with reason: testing PXE boot issues [production]
04:03 <mwpresync@deploy1002> Finished scap: testwikis wikis to 1.43.0-wmf.1 refs T361395 (duration: 57m 31s) [production]
03:05 <mwpresync@deploy1002> Started scap: testwikis wikis to 1.43.0-wmf.1 refs T361395 [production]
03:03 <mwpresync@deploy1002> Pruned MediaWiki: 1.42.0-wmf.24 (duration: 03m 11s) [production]
2024-04-15 §
17:23 <taavi@deploy1002> Finished scap: Backport for [[gerrit:1019825|wmf-config: add private subnets for magru (T346722)]] (duration: 17m 21s) [production]
17:08 <taavi@deploy1002> taavi and sukhe: Backport for [[gerrit:1019825|wmf-config: add private subnets for magru (T346722)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
17:06 <taavi@deploy1002> Started scap: Backport for [[gerrit:1019825|wmf-config: add private subnets for magru (T346722)]] [production]
2024-04-12 §
15:56 <sukhe@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on cp1115.eqiad.wmnet with reason: testing PXE boot issues [production]
15:56 <sukhe@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on cp1115.eqiad.wmnet with reason: testing PXE boot issues [production]
2024-04-11 §
17:50 <dancy@deploy1002> rebuilt and synchronized wikiversions files: group2 wikis to 1.42.0-wmf.26 refs T360158 [production]
13:13 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2129 (re)pooling @ 4%: Post upgrade', diff saved to https://phabricator.wikimedia.org/P60402 and previous config saved to /var/cache/conftool/dbconfig/20240411-131301-arnaudb.json [production]
12:53 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db[2132,2160].codfw.wmnet with reason: reboot [production]
12:53 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db[2132,2160].codfw.wmnet with reason: reboot [production]
2024-04-10 §
17:46 <swfrench-wmf> finished updating A:conf hosts to etcd-mirror 0.0.11-1 (T358636) [production]
17:37 <swfrench-wmf> restarting etcd-mirror on conf2005.codfw.wmnet for T358636 [production]
16:12 <swfrench-wmf> uploaded etcd-mirror 0.0.11-1 to apt.wikimedia.org (T358636) [production]
08:34 <hashar@deploy1002> Synchronized php: group1 wikis to 1.42.0-wmf.26 refs T360158 (duration: 13m 05s) [production]
08:21 <hashar@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.42.0-wmf.26 refs T360158 [production]
07:29 <akosiaris@deploy1002> Synchronized wmf-config/mc.php: Dummy sync for https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1018332 (duration: 14m 03s) [production]
07:07 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2212 (re)pooling @ 4%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P60197 and previous config saved to /var/cache/conftool/dbconfig/20240410-070745-arnaudb.json [production]
2024-04-09 §
09:40 <hashar@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.42.0-wmf.26 refs T360158 [production]
09:18 <hashar@deploy1002> Finished scap: testwikis wikis to 1.42.0-wmf.26 refs T360158 (duration: 50m 11s) [production]
08:52 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2113 (re)pooling @ 4%: Post reimage repool', diff saved to https://phabricator.wikimedia.org/P60002 and previous config saved to /var/cache/conftool/dbconfig/20240409-085238-arnaudb.json [production]
08:28 <hashar@deploy1002> Started scap: testwikis wikis to 1.42.0-wmf.26 refs T360158 [production]
2024-04-08 §
23:25 <tstarling@deploy1002> Synchronized wmf-config/CommonSettings.php: stop writing to ipblocks table T355034 (duration: 12m 32s) [production]
2024-04-04 §
15:00 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2213 (re)pooling @ 4%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59501 and previous config saved to /var/cache/conftool/dbconfig/20240404-150032-arnaudb.json [production]
15:00 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2207 (re)pooling @ 4%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59500 and previous config saved to /var/cache/conftool/dbconfig/20240404-150017-arnaudb.json [production]
2024-04-03 §
15:04 <jynus@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2098.codfw.wmnet with reason: restart of mysqld [production]
15:03 <jynus@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2098.codfw.wmnet with reason: restart of mysqld [production]
08:33 <jnuche@deploy1002> Synchronized php: group1 wikis to 1.42.0-wmf.25 refs T360157 (duration: 13m 00s) [production]
2024-04-02 §
17:13 <Dreamy_Jazz> Creating cu_useragent table on WMF wikis - T359312 [production]
15:35 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2220 (re)pooling @ 4%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59197 and previous config saved to /var/cache/conftool/dbconfig/20240402-153550-arnaudb.json [production]