51-100 of 1149 results (15ms)
2025-03-18 §
15:34 <hnowlan@cumin2002> END (PASS) - Cookbook sre.discovery.datacenter (exit_code=0) depool all services in codfw: Datacenter Switchover - T385155 [production]
15:05 <hnowlan@cumin2002> START - Cookbook sre.discovery.datacenter depool all services in codfw: Datacenter Switchover - T385155 [production]
14:42 <hnowlan@cumin2002> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool site codfw [reason: Datacentre switchover, T387444] [production]
14:42 <hnowlan@cumin2002> START - Cookbook sre.dns.admin DNS admin: depool site codfw [reason: Datacentre switchover, T387444] [production]
2025-03-14 §
12:03 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) for datacenter switchover from eqiad to codfw [production]
12:00 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.08-start-maintenance for datacenter switchover from eqiad to codfw [production]
11:52 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) for datacenter switchover from eqiad to codfw [production]
11:52 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.01-stop-maintenance for datacenter switchover from eqiad to codfw [production]
11:40 <hnowlan@cumin2002> END (FAIL) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=99) for datacenter switchover from eqiad to codfw [production]
11:40 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.01-stop-maintenance for datacenter switchover from eqiad to codfw [production]
2025-03-11 §
07:00 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2160,2233].codfw.wmnet,db[1164,1217,1228].eqiad.wmnet with reason: Primary switchover m2 T388396 [production]
2025-03-10 §
08:07 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2160,2232].codfw.wmnet,db[1164,1217,1250].eqiad.wmnet with reason: Primary switchover m1 T388024 [production]
2025-03-06 §
06:11 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 32 hosts with reason: Primary switchover s8 T388093 [production]
2025-03-05 §
06:26 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 33 hosts with reason: Primary switchover s4 T387816 [production]
2025-03-04 §
06:27 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 34 hosts with reason: Primary switchover s1 T387552 [production]
2025-03-03 §
12:16 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 13 hosts with reason: Primary switchover x1 T387557 [production]
2025-02-27 §
17:57 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.09-run-puppet-on-db-masters (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:46 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.09-run-puppet-on-db-masters for datacenter switchover from eqiad to codfw [production]
17:46 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.09-restore-ttl (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:45 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.09-restore-ttl for datacenter switchover from eqiad to codfw [production]
17:45 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:42 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.08-start-maintenance for datacenter switchover from eqiad to codfw [production]
17:42 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.08-restart-mw-jobrunner (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:41 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.08-restart-mw-jobrunner for datacenter switchover from eqiad to codfw [production]
17:36 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:36 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.07-set-readwrite for datacenter switchover from eqiad to codfw [production]
17:36 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:36 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite for datacenter switchover from eqiad to codfw [production]
17:36 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:35 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki for datacenter switchover from eqiad to codfw [production]
17:35 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.03-set-db-readonly (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:34 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.03-set-db-readonly for datacenter switchover from eqiad to codfw [production]
17:34 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.02-set-readonly (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:34 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.02-set-readonly for datacenter switchover from eqiad to codfw [production]
17:33 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:32 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.01-stop-maintenance for datacenter switchover from eqiad to codfw [production]
17:32 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:26 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.00-reduce-ttl for datacenter switchover from eqiad to codfw [production]
17:26 <hnowlan@cumin2002> END (FAIL) - Cookbook sre.switchdc.mediawiki.00-optional-warmup-caches (exit_code=99) for datacenter switchover from eqiad to codfw [production]
17:26 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.00-optional-warmup-caches for datacenter switchover from eqiad to codfw [production]
17:26 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.00-downtime-db-readonly-checks (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:25 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.00-downtime-db-readonly-checks for datacenter switchover from eqiad to codfw [production]
17:25 <hnowlan@cumin2002> END (PASS) - Cookbook sre.switchdc.mediawiki.00-disable-puppet (exit_code=0) for datacenter switchover from eqiad to codfw [production]
17:25 <hnowlan@cumin2002> START - Cookbook sre.switchdc.mediawiki.00-disable-puppet for datacenter switchover from eqiad to codfw [production]
13:02 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 24 hosts with reason: Primary switchover s6 T387433 [production]
2025-02-26 §
12:06 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 6 hosts with reason: Primary switchover es6 T387273 [production]
07:26 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 6 hosts with reason: Primary switchover es7 T387271 [production]
06:25 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 27 hosts with reason: Primary switchover s7 T387270 [production]
2025-02-25 §
16:12 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 6 hosts with reason: Primary switchover es7 T387224 [production]
15:51 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 6 hosts with reason: Primary switchover es7 T387224 [production]