551-600 of 10000 results (18ms)
2025-06-26 §
07:43 <jynus@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on backup1009.eqiad.wmnet with reason: Maintenance and reboot [production]
2025-06-25 §
18:41 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.45.0-wmf.7 refs T392177 [production]
16:33 <fceratto@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db1167 gradually with 4 steps - Pooling in [production]
16:02 <fceratto@cumin1002> START - Cookbook sre.mysql.pool db1167 gradually with 4 steps - Pooling in [production]
15:45 <fceratto@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1016,1020].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
14:48 <swfrench-wmf> incrementally restarting confds in codfw, ulsfo, eqsin - T352245 [production]
14:39 <swfrench-wmf> reverted etcd tlsproxy to cergen certs on conf2006 - T352245 [production]
13:49 <swfrench-wmf> restarting confd in ulsfo - T352245 [production]
13:26 <swfrench-wmf> disabled puppet on 'P{O:configcluster}' hosts - T352245 [production]
11:38 <jynus@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on backup1008.eqiad.wmnet with reason: Maintenance and reboot [production]
09:53 <jynus@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on backup2008.codfw.wmnet with reason: Maintenance and reboot [production]
2025-06-24 §
18:58 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.45.0-wmf.7 refs T392177 [production]
17:25 <swfrench-wmf> serving ~ 5% of mw-api-ext and mw-web traffic in eqiad via bookworm-based httpd image - T378128 [production]
17:11 <swfrench-wmf> serving ~ 5% of mw-api-ext and mw-web traffic in codfw via bookworm-based httpd image - T378128 [production]
14:04 <jelto> bump kubernetes-client to newest version on wikikube-ctrl100[1-4] - T387548 [production]
13:45 <tchanders@deploy1003> Finished scap sync-world: Backport for [[gerrit:1163354|Revert^2 "Enable temporary accounts onboarding dialog on WMF wikis" (T395933)]] (duration: 11m 48s) [production]
13:35 <tchanders@deploy1003> tchanders: Backport for [[gerrit:1163354|Revert^2 "Enable temporary accounts onboarding dialog on WMF wikis" (T395933)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
13:33 <tchanders@deploy1003> Started scap sync-world: Backport for [[gerrit:1163354|Revert^2 "Enable temporary accounts onboarding dialog on WMF wikis" (T395933)]] [production]
04:03 <mwpresync@deploy1003> Pruned MediaWiki: 1.45.0-wmf.3, 1.45.0-wmf.4 (duration: 03m 08s) [production]
03:49 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.45.0-wmf.7 refs T392177 (duration: 46m 40s) [production]
03:03 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.45.0-wmf.7 refs T392177 [production]
2025-06-23 §
10:41 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db2155 gradually with 4 steps - Work done [production]
09:56 <ladsgroup@cumin1002> START - Cookbook sre.mysql.pool db2155 gradually with 4 steps - Work done [production]
09:48 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db1255 gradually with 4 steps - Work done [production]
09:48 <ladsgroup@cumin1002> START - Cookbook sre.mysql.pool db1255 gradually with 4 steps - Work done [production]
2025-06-20 §
10:11 <jynus@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2184.codfw.wmnet with reason: mariadb upgrade [production]
09:24 <jynus@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on backup2009.codfw.wmnet with reason: Maintenance and reboot [production]
2025-06-19 §
08:11 <hashar@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.45.0-wmf.6 refs T392176 [production]
2025-06-18 §
17:58 <swfrench-wmf> migrated all shellbox instances to bookworm-based httpd images in eqiad - T378128 [production]
17:39 <swfrench-wmf> migrated all shellbox instances to bookworm-based httpd images in codfw - T378128 [production]
15:59 <swfrench-wmf> deployed conftool 5.3.0 to all bullseye and bookworm hosts - T395696 [production]
14:50 <swfrench-wmf> reprepro included conftool 5.3.0 in apt.wikimedia.org - T395696 [production]
14:13 <jnuche@deploy1003> Installation of scap version "4.178.3" completed for 4 hosts [production]
14:10 <jnuche@deploy1003> Installing scap version "4.178.3" for 4 host(s) [production]
13:39 <jnuche@deploy1003> Installation of scap version "4.178.2" completed for 4 hosts [production]
13:36 <jnuche@deploy1003> Installing scap version "4.178.2" for 4 host(s) [production]
13:10 <jnuche@deploy1003> Installation of scap version "4.178.1" completed for 4 hosts [production]
13:07 <jnuche@deploy1003> Installing scap version "4.178.1" for 4 host(s) [production]
10:20 <jynus@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on backup[1001,1014].eqiad.wmnet with reason: Backup director migration [production]
08:50 <hashar@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.45.0-wmf.6 refs T392176 [production]
07:41 <ryankemper@cumin2002> END (ERROR) - Cookbook sre.wdqs.data-reload (exit_code=97) reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20250526/ using stat1009.eqiad.wmnet) [production]
2025-06-17 §
18:18 <brennen@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.45.0-wmf.6 refs T392176 [production]
17:19 <swfrench-wmf> migrated shellbox-syntaxhighlight to bookworm-based httpd images - T378128 [production]
15:28 <jynus@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on ms-backup2001.codfw.wmnet with reason: Maintenance and reboot [production]
08:04 <moritzm> installing mariadb security updates (as shipped in Debian, not the wmf-mariadb packages we use for the main mariadb clusters) [production]
04:47 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.45.0-wmf.6 refs T392176 (duration: 104m 43s) [production]
03:03 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.45.0-wmf.6 refs T392176 [production]
2025-06-16 §
15:30 <dreamyjazz@deploy1003> Finished scap sync-world: Backport for [[gerrit:1159477|Revert "Enable temporary accounts onboarding dialog on WMF wikis"]] (duration: 24m 48s) [production]
15:11 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-reload reloading wikidata_main on wdqs1022.eqiad.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/main/20250526/ using stat1009.eqiad.wmnet) [production]
15:09 <dreamyjazz@deploy1003> dreamyjazz: Backport for [[gerrit:1159477|Revert "Enable temporary accounts onboarding dialog on WMF wikis"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]