|
2021-06-30
§
|
| 10:31 |
<godog> |
add 200G to prometheus/eqiad for 'ops' instance |
[production] |
| 09:35 |
<godog> |
start swiftrepl-mw on ms-fe2005 post-switchover (credentials were missing) - T162123 |
[production] |
| 08:51 |
<jelto> |
jelto@puppetmaster1001:~$ sudo puppet cert -s gitlab2001.wikimedia.org # approve puppet certificate request for gitlab2001, fingerprint checked |
[production] |
| 08:47 |
<topranks> |
Removing BGP peers for AS48237 (Etihad Etisalat) and AS11404 (Wave Division Holdings) from cr2-eqiad (peers have left Equinix IX) |
[production] |
| 08:31 |
<godog> |
remove sdf1 from thanos-be1003 in swift - T285835 |
[production] |
| 07:43 |
<filippo@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host thanos-be1003.eqiad.wmnet |
[production] |
| 07:43 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host thanos-be1003.eqiad.wmnet |
[production] |
| 07:43 |
<filippo@cumin1001> |
END (ERROR) - Cookbook sre.hosts.reboot-single (exit_code=97) for host thanos-be1003.eqiad.wmnet |
[production] |
| 07:37 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host thanos-be1003.eqiad.wmnet |
[production] |
| 05:46 |
<ryankemper> |
[Cirrus] Unbanned `elastic2045`; now only `elastic2033` is banned in `codfw` |
[production] |
| 00:36 |
<tstarling@deploy1002> |
Synchronized wmf-config/db-labs.php: gerrit 701995 SQL query log (duration: 01m 05s) |
[production] |
| 00:35 |
<tstarling@deploy1002> |
Synchronized wmf-config/db-eqiad.php: gerrit 701995 SQL query log (duration: 01m 06s) |
[production] |
| 00:34 |
<tstarling@deploy1002> |
Synchronized wmf-config/db-codfw.php: gerrit 701995 SQL query log (duration: 01m 06s) |
[production] |
| 00:32 |
<tstarling@deploy1002> |
Synchronized wmf-config/CommonSettings.php: gerrit 701995 SQL query log (duration: 01m 05s) |
[production] |
| 00:31 |
<tstarling@deploy1002> |
Synchronized docroot/noc/db.php: gerrit 701995 SQL query log (duration: 01m 06s) |
[production] |
| 00:27 |
<tstarling@deploy1002> |
Synchronized wmf-config/logging.php: gerrit 701995 SQL query log (duration: 01m 15s) |
[production] |
| 00:01 |
<urbanecm> |
(following up previous SAL item) TrainBranchBot was removed from wmf-deployment group because of T285819 |
[production] |
|
2021-06-29
§
|
| 23:45 |
<urbanecm> |
Evening B&C window done |
[production] |
| 23:45 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 367bc98: 904d18720: flood flag changes for enwikibooks (T285594) (duration: 01m 07s) |
[production] |
| 23:45 |
<urbanecm> |
Remove TrainBranchBot from wmf-deployment Gerrit group, merges code to mediawiki-config without actually deploying it |
[production] |
| 23:41 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.11/extensions/GrowthExperiments/includes/Specials/SpecialEditGrowthConfig.php: 8a5b835050cc0f6d47b6fde2317db8743fcb9ce0: SpecialEditGrowthConfig: Do not use relative => true (T285750) (duration: 01m 04s) |
[production] |
| 23:34 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.12/extensions/GrowthExperiments/includes/Specials/SpecialEditGrowthConfig.php: c61fb175c82accda526105cc32457b07530d09fa: SpecialEditGrowthConfig: Do not use relative => true (T285750) (duration: 01m 05s) |
[production] |
| 23:33 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.11/extensions/DiscussionTools/: bad82665f8bea667aff049794612c270063c7519: Config option to enable topic subscriptions backend and dtenable=1 URL parameter (T284491) (duration: 01m 05s) |
[production] |
| 23:31 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.12/extensions/DiscussionTools/: bad82665f8bea667aff049794612c270063c7519: Config option to enable topic subscriptions backend and dtenable=1 URL parameter (T284491) (duration: 01m 06s) |
[production] |
| 23:30 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.12/extensions/DiscussionTools/: e77e002130a7815570ff967013757bacc7037fb0: Config option to enable topic subscriptions backend and dtenable=1 URL parameter (T284491) (duration: 01m 09s) |
[production] |
| 21:58 |
<maryum> |
deployed security patch T285515 to wmf.12 |
[production] |
| 21:51 |
<maryum> |
deployed security patch T285515 to wmf.11 |
[production] |
| 21:44 |
<maryum> |
deployed updated security patch for T285190 to wmf.12 |
[production] |
| 21:42 |
<maryum> |
deployed updated security patch for T285190 to wmf.11 |
[production] |
| 21:31 |
<sbassett> |
Reverted and deployed updated security patch for T285190 to wmf.12 |
[production] |
| 21:29 |
<sbassett> |
Reverted and deployed updated security patch for T285190 to wmf.11 |
[production] |
| 21:19 |
<sbassett> |
Deployed updated security patch for T285190 to wmf.11 |
[production] |
| 20:55 |
<dancy> |
Deleted all CDB files on beta so they'll be recreated on the next scap sync-world run |
[production] |
| 20:26 |
<dancy> |
Reverting to scap 3.17.1-1+0~20210419163335.8~1.gbpa6b2e0 in beta |
[production] |
| 19:43 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1020.eqiad.wmnet with reason: REIMAGE |
[production] |
| 19:41 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1020.eqiad.wmnet with reason: REIMAGE |
[production] |
| 19:35 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1019.eqiad.wmnet with reason: REIMAGE |
[production] |
| 19:33 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1019.eqiad.wmnet with reason: REIMAGE |
[production] |
| 19:28 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1018.eqiad.wmnet with reason: REIMAGE |
[production] |
| 19:26 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1018.eqiad.wmnet with reason: REIMAGE |
[production] |
| 19:17 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1017.eqiad.wmnet with reason: REIMAGE |
[production] |
| 19:14 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1017.eqiad.wmnet with reason: REIMAGE |
[production] |
| 19:07 |
<brennen@deploy1002> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.37.0-wmf.12 |
[production] |
| 18:57 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1016.eqiad.wmnet with reason: REIMAGE |
[production] |
| 18:55 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1016.eqiad.wmnet with reason: REIMAGE |
[production] |
| 18:34 |
<Krinkle> |
krinkle@mwmaint2002.codfw: mwscript purgeParserCache.php --wiki=aawiki --age=1814400 --msleep 200 --tag pc3 |
[production] |
| 18:28 |
<Krinkle> |
krinkle@mwmaint2002.codfw: mwscript purgeParserCache.php --wiki=aawiki --age=1814400 --msleep 200 --tag pc2 |
[production] |
| 18:21 |
<Krinkle> |
krinkle@mwmaint2002.codfw: mwscript purgeParserCache.php --wiki=aawiki --age=1814400 --msleep 200 --tag pc1 |
[production] |
| 18:11 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1016.eqiad.wmnet with reason: REIMAGE |
[production] |
| 18:09 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1016.eqiad.wmnet with reason: REIMAGE |
[production] |