|
2021-08-13
§
|
| 10:22 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on mw1444.eqiad.wmnet with reason: new setup |
[production] |
| 10:07 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=thanos-fe2003.codfw.wmnet |
[production] |
| 09:53 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on mw1444.eqiad.wmnet with reason: new setup |
[production] |
| 09:53 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime for 4:00:00 on mw1444.eqiad.wmnet with reason: new setup |
[production] |
| 09:42 |
<mutante> |
mw1448, mw1449, mw1450 - powering on via mgmt - OS install, initial setup (T279309, T273915) |
[production] |
| 09:38 |
<filippo@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on thanos-fe2003.codfw.wmnet with reason: REIMAGE |
[production] |
| 09:35 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on thanos-fe2003.codfw.wmnet with reason: REIMAGE |
[production] |
| 09:35 |
<mutante> |
mw1444 - signed puppet cert, initial run (after hardware fix) T279309 |
[production] |
| 09:17 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=thanos-fe2003.codfw.wmnet |
[production] |
| 09:17 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=thanos-fe2001.codfw.wmnet |
[production] |
| 09:15 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=thanos-fe2002.codfw.wmnet |
[production] |
| 08:42 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on thanos-fe2002.codfw.wmnet with reason: REIMAGE |
[production] |
| 08:40 |
<filippo@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=thanos-fe2002.codfw.wmnet |
[production] |
| 08:40 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on thanos-fe2002.codfw.wmnet with reason: REIMAGE |
[production] |
| 05:24 |
<marostegui@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1132.eqiad.wmnet with reason: REIMAGE |
[production] |
| 05:22 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db1132.eqiad.wmnet with reason: REIMAGE |
[production] |
| 01:02 |
<tgr> |
running extensions/GrowthExperiments/maintenance/fixLinkRecommendationData.php for Growth wikis |
[production] |
|
2021-08-12
§
|
| 23:50 |
<derick@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:712732|Set archive namespaces on foundationwiki to 'noindex,follow' (T288763)]] (duration: 00m 59s) |
[production] |
| 23:47 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 23:45 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 23:38 |
<cjming@deploy1002> |
Synchronized php-1.37.0-wmf.18/extensions/GrowthExperiments: Backport: [[gerrit:711719|Add Link: fix invalidation on non-addlink edit (T283606)]] (duration: 01m 00s) |
[production] |
| 23:34 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 23:33 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 22:09 |
<tgr> |
T283867 running userOptions.php on Growth wikis as per T283867#7280296 |
[production] |
| 22:00 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 21:59 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 21:57 |
<ladsgroup@deploy1002> |
Synchronized php-1.37.0-wmf.18/extensions/SpamBlacklist/includes/SpamBlacklistHooks.php: Backport: [[gerrit:711721|Don't generate HTML when asking for ParserOutput (T288639)]] (duration: 00m 58s) |
[production] |
| 21:52 |
<urbanecm> |
Run `mwscript extensions/Translate/scripts/refresh-translatable-pages.php --wiki=$WIKI --jobqueue` for a bunch of Translate-enabled wikis (T288683) |
[production] |
| 21:32 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 21:31 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 21:30 |
<twentyafterfour@deploy1002> |
rebuilt and synchronized wikiversions files: all wikis to 1.37.0-wmf.18 refs T281159 |
[production] |
| 21:13 |
<twentyafterfour@deploy1002> |
Synchronized php-1.37.0-wmf.18/extensions/DiscussionTools/includes/Notifications/EventDispatcher.php: sync Ic27418a0ec976347be5fa586bbd32cc4a0d8d511 to unblock the train refs T288775 and T281159 (duration: 01m 07s) |
[production] |
| 20:56 |
<urbanecm> |
[urbanecm@mwmaint2002 /srv/mediawiki/php]$ mwscript extensions/Translate/scripts/refresh-translatable-pages.php --wiki=testwikidatawiki --jobqueue # T288683, errored out |
[production] |
| 20:54 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 20:54 |
<urbanecm> |
[urbanecm@mwmaint2002 /srv/mediawiki/php]$ mwscript extensions/Translate/scripts/refresh-translatable-pages.php --wiki=testwiki --jobqueue # T288683 |
[production] |
| 20:53 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 20:24 |
<urbanecm> |
[urbanecm@mwmaint2002 /srv/mediawiki/php]$ mwscript extensions/Translate/scripts/refresh-translatable-pages.php --wiki=wikimaniawiki --jobqueue # T288683 |
[production] |
| 20:13 |
<urbanecm> |
[urbanecm@mwmaint2002 /srv/mediawiki/php]$ mwscript extensions/Translate/scripts/refresh-translatable-pages.php --wiki=wikimaniawiki --jobqueue # T288683 |
[production] |
| 19:43 |
<twentyafterfour@deploy1002> |
Synchronized php-1.37.0-wmf.18/extensions/Translate/src/PageTranslation/TranslationPage.php: sync I2f46abb20145630c27449ce57f1256e92f440144 which should fix T288683 & T288700 thus unblocking the train: T281159 (duration: 01m 07s) |
[production] |
| 19:36 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:34 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 18:52 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 18:49 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 16:47 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host doh4002.wikimedia.org |
[production] |
| 16:37 |
<dzahn@cumin1001> |
START - Cookbook sre.ganeti.makevm for new host doh4002.wikimedia.org |
[production] |
| 16:32 |
<mbsantos@deploy1002> |
Finished deploy [tilerator/deploy@b88cf50]: maps1005: (duration: 00m 15s) |
[production] |
| 16:32 |
<mbsantos@deploy1002> |
Started deploy [tilerator/deploy@b88cf50]: maps1005: |
[production] |
| 16:32 |
<effie> |
enabling puppet on mediawiki servers && rolling restart mcrouter |
[production] |
| 16:31 |
<mbsantos@deploy1002> |
Finished deploy [tilerator/deploy@b88cf50]: maps1006: (duration: 00m 15s) |
[production] |
| 16:31 |
<mbsantos@deploy1002> |
Started deploy [tilerator/deploy@b88cf50]: maps1006: |
[production] |