|
2020-03-03
§
|
| 09:07 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Add es4 to the available es sections, not in use yet - T246072 (duration: 00m 57s) |
[production] |
| 08:53 |
<addshore> |
START warm cache for db1111 & db1126 for Q12-15 million T219123 (pass 3) |
[production] |
| 08:36 |
<elukey@cumin1001> |
START - Cookbook sre.kafka.roll-restart-brokers |
[production] |
| 08:25 |
<addshore> |
addshore@mwmaint1002:~$ time mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=wikidatawiki --batch-size=25 --sleep=1 --file=27feb1125-40to50-holes # T219123 |
[production] |
| 08:13 |
<addshore> |
START warm cache for db1111 & db1126 for Q12-15 million T219123 (pass 2) |
[production] |
| 08:11 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.zookeeper.roll-restart-zookeeper (exit_code=0) |
[production] |
| 08:08 |
<addshore> |
addshore@mwmaint1002:~$ time mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=wikidatawiki --batch-size=25 --sleep=1 --file=27feb1125-30to40-holes # T219123 |
[production] |
| 08:05 |
<elukey@cumin1001> |
START - Cookbook sre.zookeeper.roll-restart-zookeeper |
[production] |
| 07:55 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.zookeeper.roll-restart-zookeeper (exit_code=0) |
[production] |
| 07:48 |
<elukey@cumin1001> |
START - Cookbook sre.zookeeper.roll-restart-zookeeper |
[production] |
| 07:45 |
<addshore> |
START warm cache for db1111 & db1126 for Q12-15 million T219123 (pass 1) |
[production] |
| 07:41 |
<vgutierrez> |
Re-enable BGP in lvs3006 - T245984 |
[production] |
| 07:39 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.druid.roll-restart-workers (exit_code=0) |
[production] |
| 07:19 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 07:17 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 07:09 |
<elukey@cumin1001> |
START - Cookbook sre.druid.roll-restart-workers |
[production] |
| 07:09 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 07:07 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.druid.roll-restart-workers (exit_code=0) |
[production] |
| 07:07 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 06:51 |
<vgutierrez> |
reimage lvs3006 with buster - T245984 |
[production] |
| 06:49 |
<marostegui> |
Stop MySQL on db1096:3315,3316 for reimage - T246604 |
[production] |
| 06:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1096:3315, db1096:3316 for reimage T246604', diff saved to https://phabricator.wikimedia.org/P10587 and previous config saved to /var/cache/conftool/dbconfig/20200303-064316-marostegui.json |
[production] |
| 06:41 |
<elukey@cumin1001> |
START - Cookbook sre.druid.roll-restart-workers |
[production] |
| 06:33 |
<vgutierrez> |
Switch from globalsign to LE as unified cert vendor on eqiad - T230687 |
[production] |
| 06:25 |
<vgutierrez> |
Switch from globalsign to LE as unified cert vendor on codfw - T230687 |
[production] |
| 02:57 |
<mutante> |
manually running "Ancientpages" cron on s3 (T243599) |
[production] |
| 02:39 |
<mutante> |
manually running updateSpecialPages.php maintenance cron on s8 for AncientPages to confirm it was fixed by gerrit:574726 a few days ago (T243599) |
[production] |
| 00:53 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T240055 Update special scandium configuration to load from /srv/parsoid-testing (duration: 00m 58s) |
[production] |
| 00:36 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Touch and secondary sync of IS for cache-busting (duration: 00m 56s) |
[production] |
| 00:35 |
<jforrester@deploy1001> |
Synchronized dblists/mobilemainpagelegacy.dblist: T32405 Drop legacy main page special casing on select projects (duration: 00m 56s) |
[production] |
| 00:31 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Touch and secondary sync of IS for cache-busting (duration: 00m 56s) |
[production] |
| 00:28 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T242030 Enable lead paragraph in user namespace on nlwiki (duration: 00m 56s) |
[production] |
| 00:15 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp1025.eqiad.wmnet |
[production] |
| 00:14 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T240055: Point the Parsoid cluster at the train version of Parsoid, not a special check-out (duration: 00m 56s) |
[production] |
| 00:13 |
<jforrester@deploy1001> |
sync aborted: wmf-config/CommonSettings.php T240055: Point the Parsoid cluster at the train version of Parsoid, not a special check-out (duration: 00m 03s) |
[production] |
| 00:13 |
<jforrester@deploy1001> |
Started scap: wmf-config/CommonSettings.php T240055: Point the Parsoid cluster at the train version of Parsoid, not a special check-out |
[production] |
| 00:05 |
<mutante> |
wtp1025 - scap pull |
[production] |
|
2020-03-02
§
|
| 23:58 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=wtp1025.eqiad.wmnet |
[production] |
| 23:43 |
<krinkle@deploy1001> |
Synchronized docroot/noc/: Idc26716abef5bff (duration: 00m 56s) |
[production] |
| 23:42 |
<krinkle@deploy1001> |
Synchronized multiversion/: Idc26716abef5bff (duration: 00m 57s) |
[production] |
| 23:41 |
<krinkle@deploy1001> |
Synchronized src/: Idc26716abef5bff (duration: 00m 56s) |
[production] |
| 23:36 |
<krinkle@deploy1001> |
Synchronized wmf-config/CommonSettings.php: I1bdc5e359476 (duration: 00m 56s) |
[production] |
| 21:10 |
<XioNoX> |
re-number AMS-IX peer 64271 |
[production] |
| 20:22 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: Enable Mediawiki client side error logging on group0 wikis - T246030 (duration: 00m 57s) |
[production] |
| 20:12 |
<krinkle@deploy1001> |
Synchronized multiversion/MWConfigCacheGenerator.php: I1ef0589 (duration: 00m 58s) |
[production] |
| 20:01 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: Enable Mediawiki client side error logging in beta - T246030 (duration: 00m 56s) |
[production] |
| 19:35 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: EventStreamConfig - allow eventgate-analytics-external to produce error events - T233629 (duration: 00m 56s) |
[production] |
| 19:17 |
<ppchelko@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Enable REST run jobs endpoint on jobrunners T244770 (duration: 00m 56s) |
[production] |
| 19:15 |
<ppchelko@deploy1001> |
Synchronized wmf-config/CommonSettings-labs.php: Enable REST run jobs endpoint on jobrunners T244770 (duration: 00m 56s) |
[production] |
| 19:04 |
<otto@deploy1001> |
Synchronized wmf-config/ProductionServices.php: Use new LVS port for EventBus for eventgate-main on all wikis - T245203 (duration: 00m 56s) |
[production] |