|
2024-02-01
§
|
| 06:07 |
<marostegui> |
Starting es4 codfw failover from es2024 to es2023 - T356235 |
[production] |
| 06:02 |
<marostegui@deploy2002> |
Finished scap: Backport for [[gerrit:994611|db-production.php: Disable writes on es5 (T356235)]] (duration: 08m 19s) |
[production] |
| 05:59 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2099.codfw.wmnet with reason: Maintenance |
[production] |
| 05:58 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2099.codfw.wmnet with reason: Maintenance |
[production] |
| 05:55 |
<marostegui@deploy2002> |
marostegui: Continuing with sync |
[production] |
| 05:55 |
<marostegui@deploy2002> |
marostegui: Backport for [[gerrit:994611|db-production.php: Disable writes on es5 (T356235)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 05:53 |
<marostegui@deploy2002> |
Started scap: Backport for [[gerrit:994611|db-production.php: Disable writes on es5 (T356235)]] |
[production] |
| 05:52 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Set es2023 with weight 0 T356235', diff saved to https://phabricator.wikimedia.org/P56011 and previous config saved to /var/cache/conftool/dbconfig/20240201-055240-marostegui.json |
[production] |
| 05:52 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 6 hosts with reason: Primary switchover es5 T356235 |
[production] |
| 05:52 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 1:00:00 on 6 hosts with reason: Primary switchover es5 T356235 |
[production] |
| 02:55 |
<ejegg> |
civicrm upgraded from 6e1e0d21 to 21bf2138 |
[production] |
| 00:20 |
<urbanecm@deploy2002> |
Finished scap: Backport for [[gerrit:994857|testwiki: Enable conditional defaults for 4 Echo properties (T353225)]] (duration: 08m 09s) |
[production] |
| 00:13 |
<urbanecm@deploy2002> |
urbanecm: Continuing with sync |
[production] |
| 00:13 |
<urbanecm@deploy2002> |
urbanecm: Backport for [[gerrit:994857|testwiki: Enable conditional defaults for 4 Echo properties (T353225)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 00:12 |
<urbanecm@deploy2002> |
Started scap: Backport for [[gerrit:994857|testwiki: Enable conditional defaults for 4 Echo properties (T353225)]] |
[production] |
|
2024-01-31
§
|
| 23:11 |
<eileen> |
* civicrm upgraded from 6344c95e to 6e1e0d21 |
[production] |
| 22:29 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:29 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 22:28 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1233 (T355609)', diff saved to https://phabricator.wikimedia.org/P56010 and previous config saved to /var/cache/conftool/dbconfig/20240131-222853-marostegui.json |
[production] |
| 22:13 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1233', diff saved to https://phabricator.wikimedia.org/P56009 and previous config saved to /var/cache/conftool/dbconfig/20240131-221347-marostegui.json |
[production] |
| 22:11 |
<jdrewniak@deploy2002> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:994823| Bumping portals to master (T128546)]] (duration: 06m 43s) |
[production] |
| 22:05 |
<jdrewniak@deploy2002> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:994823| Bumping portals to master (T128546)]] (duration: 07m 26s) |
[production] |
| 21:58 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1233', diff saved to https://phabricator.wikimedia.org/P56008 and previous config saved to /var/cache/conftool/dbconfig/20240131-215840-marostegui.json |
[production] |
| 21:54 |
<Dreamy_Jazz> |
Removed already applied patches for T347708 from /srv/patches |
[production] |
| 21:48 |
<dancy@deploy2002> |
Synchronized php: group1 wikis to 1.42.0-wmf.16 refs T354434 (duration: 06m 47s) |
[production] |
| 21:43 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1233 (T355609)', diff saved to https://phabricator.wikimedia.org/P56007 and previous config saved to /var/cache/conftool/dbconfig/20240131-214334-marostegui.json |
[production] |
| 21:42 |
<dancy@deploy2002> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.42.0-wmf.16 refs T354434 |
[production] |
| 21:35 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1233 (T355609)', diff saved to https://phabricator.wikimedia.org/P56006 and previous config saved to /var/cache/conftool/dbconfig/20240131-213454-marostegui.json |
[production] |
| 21:35 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1233.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:34 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1233.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:34 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1229 (T355609)', diff saved to https://phabricator.wikimedia.org/P56005 and previous config saved to /var/cache/conftool/dbconfig/20240131-213432-marostegui.json |
[production] |
| 21:31 |
<Dreamy_Jazz> |
Security deploy done |
[production] |
| 21:30 |
<logmsgbot> |
dreamyjazz Deployed security patch for T356226 |
[production] |
| 21:23 |
<logmsgbot> |
dreamyjazz Deployed security patch for T356226 |
[production] |
| 21:19 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1229', diff saved to https://phabricator.wikimedia.org/P56004 and previous config saved to /var/cache/conftool/dbconfig/20240131-211926-marostegui.json |
[production] |
| 21:16 |
<Dreamy_Jazz> |
Doing security deploy for T356226 |
[production] |
| 21:12 |
<jforrester@deploy2002> |
Finished scap: Backport for [[gerrit:994716|Gadget: Bump GADGET_CLASS_VERSION (T356322)]] (duration: 08m 31s) |
[production] |
| 21:05 |
<jforrester@deploy2002> |
jforrester and reedy: Continuing with sync |
[production] |
| 21:05 |
<jforrester@deploy2002> |
jforrester and reedy: Backport for [[gerrit:994716|Gadget: Bump GADGET_CLASS_VERSION (T356322)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:04 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1229', diff saved to https://phabricator.wikimedia.org/P56003 and previous config saved to /var/cache/conftool/dbconfig/20240131-210419-marostegui.json |
[production] |
| 21:03 |
<jforrester@deploy2002> |
Started scap: Backport for [[gerrit:994716|Gadget: Bump GADGET_CLASS_VERSION (T356322)]] |
[production] |
| 20:49 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1229 (T355609)', diff saved to https://phabricator.wikimedia.org/P56002 and previous config saved to /var/cache/conftool/dbconfig/20240131-204913-marostegui.json |
[production] |
| 20:44 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1229 (T355609)', diff saved to https://phabricator.wikimedia.org/P56001 and previous config saved to /var/cache/conftool/dbconfig/20240131-204439-marostegui.json |
[production] |
| 20:44 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1229.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:44 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1229.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:37 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1225.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:37 |
<eevans@deploy2002> |
helmfile [eqiad] DONE helmfile.d/services/sessionstore: apply |
[production] |
| 20:37 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1225.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:37 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1222 (T355609)', diff saved to https://phabricator.wikimedia.org/P56000 and previous config saved to /var/cache/conftool/dbconfig/20240131-203704-marostegui.json |
[production] |
| 20:36 |
<eevans@deploy2002> |
helmfile [eqiad] START helmfile.d/services/sessionstore: apply |
[production] |