|
2024-09-11
§
|
| 07:33 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2137 (re)pooling @ 50%: post db2137 → db2237 repool', diff saved to https://phabricator.wikimedia.org/P68894 and previous config saved to /var/cache/conftool/dbconfig/20240911-073307-arnaudb.json |
[production] |
| 07:29 |
<sgimeno@deploy1003> |
sgimeno: Continuing with sync |
[production] |
| 07:26 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2179 T374512', diff saved to https://phabricator.wikimedia.org/P68893 and previous config saved to /var/cache/conftool/dbconfig/20240911-072612-arnaudb.json |
[production] |
| 07:24 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2179 T374512', diff saved to https://phabricator.wikimedia.org/P68892 and previous config saved to /var/cache/conftool/dbconfig/20240911-072458-arnaudb.json |
[production] |
| 07:24 |
<sgimeno@deploy1003> |
sgimeno: Backport for [[gerrit:1062416|EventStreamConfig and stream registration for homepage modules analytics (T370907)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 07:22 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Promote db2140 to s4 primary T374512', diff saved to https://phabricator.wikimedia.org/P68891 and previous config saved to /var/cache/conftool/dbconfig/20240911-072210-arnaudb.json |
[production] |
| 07:21 |
<arnaudb> |
Starting s4 codfw failover from db2179 to db2140 - T374512 |
[production] |
| 07:19 |
<sgimeno@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1062416|EventStreamConfig and stream registration for homepage modules analytics (T370907)]] |
[production] |
| 07:18 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2137 (re)pooling @ 25%: post db2137 → db2237 repool', diff saved to https://phabricator.wikimedia.org/P68890 and previous config saved to /var/cache/conftool/dbconfig/20240911-071802-arnaudb.json |
[production] |
| 07:13 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Remove db2140 from API/vslow/dump T374512', diff saved to https://phabricator.wikimedia.org/P68889 and previous config saved to /var/cache/conftool/dbconfig/20240911-071335-arnaudb.json |
[production] |
| 07:12 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 33 hosts with reason: Primary switchover s4 T374512 |
[production] |
| 07:12 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Set db2140 with weight 0 T374512', diff saved to https://phabricator.wikimedia.org/P68888 and previous config saved to /var/cache/conftool/dbconfig/20240911-071205-arnaudb.json |
[production] |
| 07:11 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1:00:00 on 33 hosts with reason: Primary switchover s4 T374512 |
[production] |
| 07:02 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2137 (re)pooling @ 10%: post db2137 → db2237 repool', diff saved to https://phabricator.wikimedia.org/P68887 and previous config saved to /var/cache/conftool/dbconfig/20240911-070254-arnaudb.json |
[production] |
| 06:34 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1234', diff saved to https://phabricator.wikimedia.org/P68884 and previous config saved to /var/cache/conftool/dbconfig/20240911-063458-ladsgroup.json |
[production] |
| 06:19 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1234', diff saved to https://phabricator.wikimedia.org/P68883 and previous config saved to /var/cache/conftool/dbconfig/20240911-061951-ladsgroup.json |
[production] |
| 06:04 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1234 (T371742)', diff saved to https://phabricator.wikimedia.org/P68882 and previous config saved to /var/cache/conftool/dbconfig/20240911-060444-ladsgroup.json |
[production] |
| 05:05 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1234 (T371742)', diff saved to https://phabricator.wikimedia.org/P68881 and previous config saved to /var/cache/conftool/dbconfig/20240911-050506-ladsgroup.json |
[production] |
| 05:04 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1234.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:04 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1234.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:04 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1232 (T371742)', diff saved to https://phabricator.wikimedia.org/P68880 and previous config saved to /var/cache/conftool/dbconfig/20240911-050444-ladsgroup.json |
[production] |
| 04:49 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1232', diff saved to https://phabricator.wikimedia.org/P68879 and previous config saved to /var/cache/conftool/dbconfig/20240911-044936-ladsgroup.json |
[production] |
| 04:34 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1232', diff saved to https://phabricator.wikimedia.org/P68878 and previous config saved to /var/cache/conftool/dbconfig/20240911-043429-ladsgroup.json |
[production] |
| 04:19 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1232 (T371742)', diff saved to https://phabricator.wikimedia.org/P68877 and previous config saved to /var/cache/conftool/dbconfig/20240911-041922-ladsgroup.json |
[production] |
| 03:16 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1232 (T371742)', diff saved to https://phabricator.wikimedia.org/P68876 and previous config saved to /var/cache/conftool/dbconfig/20240911-031643-ladsgroup.json |
[production] |
| 03:16 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1232.eqiad.wmnet with reason: Maintenance |
[production] |
| 03:16 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1232.eqiad.wmnet with reason: Maintenance |
[production] |
| 03:16 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1219 (T371742)', diff saved to https://phabricator.wikimedia.org/P68875 and previous config saved to /var/cache/conftool/dbconfig/20240911-031621-ladsgroup.json |
[production] |
| 03:01 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1219', diff saved to https://phabricator.wikimedia.org/P68874 and previous config saved to /var/cache/conftool/dbconfig/20240911-030112-ladsgroup.json |
[production] |
| 02:46 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1219', diff saved to https://phabricator.wikimedia.org/P68873 and previous config saved to /var/cache/conftool/dbconfig/20240911-024605-ladsgroup.json |
[production] |
| 02:30 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1219 (T371742)', diff saved to https://phabricator.wikimedia.org/P68872 and previous config saved to /var/cache/conftool/dbconfig/20240911-023058-ladsgroup.json |
[production] |
| 01:33 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1219 (T371742)', diff saved to https://phabricator.wikimedia.org/P68871 and previous config saved to /var/cache/conftool/dbconfig/20240911-013327-ladsgroup.json |
[production] |
| 01:33 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1219.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:33 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1219.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:33 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1218 (T371742)', diff saved to https://phabricator.wikimedia.org/P68870 and previous config saved to /var/cache/conftool/dbconfig/20240911-013305-ladsgroup.json |
[production] |
| 01:17 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1218', diff saved to https://phabricator.wikimedia.org/P68869 and previous config saved to /var/cache/conftool/dbconfig/20240911-011758-ladsgroup.json |
[production] |
| 01:02 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1218', diff saved to https://phabricator.wikimedia.org/P68868 and previous config saved to /var/cache/conftool/dbconfig/20240911-010250-ladsgroup.json |
[production] |
| 00:53 |
<eileen> |
civicrm upgraded from e830e526 to 929101dc |
[production] |
| 00:47 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1218 (T371742)', diff saved to https://phabricator.wikimedia.org/P68867 and previous config saved to /var/cache/conftool/dbconfig/20240911-004743-ladsgroup.json |
[production] |
|
2024-09-10
§
|
| 23:51 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1218 (T371742)', diff saved to https://phabricator.wikimedia.org/P68866 and previous config saved to /var/cache/conftool/dbconfig/20240910-235102-ladsgroup.json |
[production] |
| 23:50 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1218.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:50 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1218.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:50 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207 (T371742)', diff saved to https://phabricator.wikimedia.org/P68865 and previous config saved to /var/cache/conftool/dbconfig/20240910-235040-ladsgroup.json |
[production] |
| 23:49 |
<cstone> |
civicrm upgraded from 5dd4edc1 to e830e526 |
[production] |
| 23:35 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P68864 and previous config saved to /var/cache/conftool/dbconfig/20240910-233533-ladsgroup.json |
[production] |
| 23:20 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P68863 and previous config saved to /var/cache/conftool/dbconfig/20240910-232026-ladsgroup.json |
[production] |
| 23:05 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207 (T371742)', diff saved to https://phabricator.wikimedia.org/P68862 and previous config saved to /var/cache/conftool/dbconfig/20240910-230518-ladsgroup.json |
[production] |
| 22:07 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1207 (T371742)', diff saved to https://phabricator.wikimedia.org/P68861 and previous config saved to /var/cache/conftool/dbconfig/20240910-220748-ladsgroup.json |
[production] |
| 22:07 |
<ladsgroup@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1207.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:07 |
<ladsgroup@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1207.eqiad.wmnet with reason: Maintenance |
[production] |