3251-3300 of 10000 results (33ms)
2024-09-16 §
15:56 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2222 (re)pooling @ 4%: T374623', diff saved to https://phabricator.wikimedia.org/P69149 and previous config saved to /var/cache/conftool/dbconfig/20240916-155612-arnaudb.json [production]
15:56 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2221 (re)pooling @ 4%: T374623', diff saved to https://phabricator.wikimedia.org/P69148 and previous config saved to /var/cache/conftool/dbconfig/20240916-155607-arnaudb.json [production]
2024-09-13 §
16:57 <swfrench-wmf> running homer cr*codfw* commit 'T372878' [production]
16:38 <swfrench-wmf> running homer lsw1-b3-codfw* commit 'T372878' [production]
10:44 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on db1172.eqiad.wmnet with reason: Depooled recovering replag [production]
10:44 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on db1172.eqiad.wmnet with reason: Depooled recovering replag [production]
2024-09-12 §
19:03 <swfrench-wmf> rebuilt php8.1 production images to pick up php-uuid - T372602 [production]
18:20 <swfrench-wmf> ran systemctl reset-failed mediawiki_job_MachineVision_prioritize_uncategorized.service on mwmaint1002 to clear failed state for turned down job - T352884 [production]
18:18 <dduvall@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.22 refs T373641 [production]
13:51 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1209.eqiad.wmnet with reason: Maintenance [production]
13:51 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1209.eqiad.wmnet with reason: Maintenance [production]
13:03 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1192.eqiad.wmnet with reason: Maintenance [production]
13:03 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1192.eqiad.wmnet with reason: Maintenance [production]
12:36 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2123.codfw.wmnet with reason: Maintenance [production]
12:36 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2123.codfw.wmnet with reason: Maintenance [production]
08:23 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2129 (re)pooling @ 4%: post db2229 bootstrap', diff saved to https://phabricator.wikimedia.org/P69022 and previous config saved to /var/cache/conftool/dbconfig/20240912-082301-arnaudb.json [production]
2024-09-11 §
18:25 <dduvall@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.22 refs T373641 [production]
17:39 <swfrench-wmf> imported php-uuid_1.2.0-12+wmf11u1 into component/php81 - T372507 [production]
2024-09-10 §
18:38 <swfrench-wmf> ran authdns-update on dns1004 (18:25 UTC) for T372604 [production]
18:14 <dduvall@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.22 refs T373641 [production]
18:04 <swfrench-wmf> ran sre.dns.netbox after adding mwdebug-next LVS VIPs for T372604 [production]
17:39 <tzatziki> removing 4 files for legal compliance [production]
15:02 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2127 (re)pooling @ 4%: post reimage && maintenance repool', diff saved to https://phabricator.wikimedia.org/P68809 and previous config saved to /var/cache/conftool/dbconfig/20240910-150222-arnaudb.json [production]
04:01 <mwpresync@deploy1003> Pruned MediaWiki: 1.43.0-wmf.19 (duration: 00m 58s) [production]
03:47 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.43.0-wmf.22 refs T373641 (duration: 45m 06s) [production]
03:02 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.22 refs T373641 [production]
2024-09-07 §
10:28 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on db1246.eqiad.wmnet with reason: https://phabricator.wikimedia.org/T374215 → server depooled has hardware issues [production]
10:28 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on db1246.eqiad.wmnet with reason: https://phabricator.wikimedia.org/T374215 → server depooled has hardware issues [production]
2024-09-06 §
10:33 <akosiaris@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1246.eqiad.wmnet with reason: Server failed, rebooted in emergency/single user mode [production]
10:32 <akosiaris@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1246.eqiad.wmnet with reason: Server failed, rebooted in emergency/single user mode [production]
2024-09-05 §
18:37 <dancy@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.21 refs T373640 [production]
16:38 <dancy@deploy1003> rebuilt and synchronized wikiversions files: group1 to 1.43.0-wmf.21 refs T373640 [production]
2024-09-04 §
21:03 <dancy@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.21 refs T373640 [production]
17:59 <dancy@deploy1003> Finished scap sync-world: testwikis to 1.43.0-wmf.21 refs T373640 (duration: 06m 48s) [production]
17:52 <dancy@deploy1003> Started scap sync-world: testwikis to 1.43.0-wmf.21 refs T373640 [production]
17:16 <dancy@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.43.0-wmf.21 refs T373640 [production]
16:13 <swfrench-wmf> running homer 'cr*codfw*' commit 'T374018' [production]
10:37 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2218.codfw.wmnet with reason: Maintenance [production]
10:36 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2218.codfw.wmnet with reason: Maintenance [production]
09:48 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1181.eqiad.wmnet with reason: Maintenance [production]
09:48 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1181.eqiad.wmnet with reason: Maintenance [production]
08:36 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
08:36 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
07:56 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2214.codfw.wmnet with reason: Maintenance [production]
07:56 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2214.codfw.wmnet with reason: Maintenance [production]
07:10 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:09 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
2024-09-03 §
21:42 <swfrench-wmf> running homer 'cr*codfw*' commit 'T372878' [production]
21:31 <swfrench-wmf> running homer 'lsw1-b3-codfw*' commit 'T372878' [production]
20:44 <swfrench-wmf> running homer 'lsw1-b3-codfw*' commit 'T372878' [production]