|
2021-04-15
§
|
| 10:08 |
<akosiaris@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'linkrecommendation' for release 'internal' . |
[production] |
| 10:08 |
<akosiaris@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . |
[production] |
| 10:08 |
<akosiaris@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'linkrecommendation' for release 'staging' . |
[production] |
| 09:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 100%: Repool db1146:3312', diff saved to https://phabricator.wikimedia.org/P15368 and previous config saved to /var/cache/conftool/dbconfig/20210415-095031-root.json |
[production] |
| 09:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 100%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15367 and previous config saved to /var/cache/conftool/dbconfig/20210415-093633-root.json |
[production] |
| 09:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 75%: Repool db1146:3312', diff saved to https://phabricator.wikimedia.org/P15366 and previous config saved to /var/cache/conftool/dbconfig/20210415-093527-root.json |
[production] |
| 09:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 75%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15365 and previous config saved to /var/cache/conftool/dbconfig/20210415-092129-root.json |
[production] |
| 09:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 50%: Repool db1146:3312', diff saved to https://phabricator.wikimedia.org/P15364 and previous config saved to /var/cache/conftool/dbconfig/20210415-092024-root.json |
[production] |
| 09:16 |
<ema> |
cp-upload: varnishadm -n frontend param.set nuke_limit 1000 T275809 |
[production] |
| 09:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 50%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15363 and previous config saved to /var/cache/conftool/dbconfig/20210415-090625-root.json |
[production] |
| 09:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 25%: Repool db1146:3312', diff saved to https://phabricator.wikimedia.org/P15362 and previous config saved to /var/cache/conftool/dbconfig/20210415-090520-root.json |
[production] |
| 09:04 |
<moritzm> |
installing tomcat security updates |
[production] |
| 08:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 25%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15361 and previous config saved to /var/cache/conftool/dbconfig/20210415-085122-root.json |
[production] |
| 08:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 10%: Repool db1146:3312', diff saved to https://phabricator.wikimedia.org/P15360 and previous config saved to /var/cache/conftool/dbconfig/20210415-085017-root.json |
[production] |
| 08:48 |
<godog> |
free space and bounce thanos-compact |
[production] |
| 08:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 10%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15359 and previous config saved to /var/cache/conftool/dbconfig/20210415-083618-root.json |
[production] |
| 08:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 5%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15358 and previous config saved to /var/cache/conftool/dbconfig/20210415-082115-root.json |
[production] |
| 08:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1146:3314', diff saved to https://phabricator.wikimedia.org/P15357 and previous config saved to /var/cache/conftool/dbconfig/20210415-081127-marostegui.json |
[production] |
| 08:09 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 100%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15356 and previous config saved to /var/cache/conftool/dbconfig/20210415-080947-root.json |
[production] |
| 07:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 100%: Repool db1166 after cloning db1179', diff saved to https://phabricator.wikimedia.org/P15355 and previous config saved to /var/cache/conftool/dbconfig/20210415-075718-root.json |
[production] |
| 07:54 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 75%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15354 and previous config saved to /var/cache/conftool/dbconfig/20210415-075444-root.json |
[production] |
| 07:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 75%: Repool db1166 after cloning db1179', diff saved to https://phabricator.wikimedia.org/P15353 and previous config saved to /var/cache/conftool/dbconfig/20210415-074214-root.json |
[production] |
| 07:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 50%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15352 and previous config saved to /var/cache/conftool/dbconfig/20210415-073940-root.json |
[production] |
| 07:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 50%: Repool db1166 after cloning db1179', diff saved to https://phabricator.wikimedia.org/P15351 and previous config saved to /var/cache/conftool/dbconfig/20210415-072711-root.json |
[production] |
| 07:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 25%: Repool db1146:3314 after kernel upgrade', diff saved to https://phabricator.wikimedia.org/P15350 and previous config saved to /var/cache/conftool/dbconfig/20210415-072436-root.json |
[production] |
| 07:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1146 (s2,s4) to upgrade kernel', diff saved to https://phabricator.wikimedia.org/P15348 and previous config saved to /var/cache/conftool/dbconfig/20210415-071600-marostegui.json |
[production] |
| 07:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 25%: Repool db1166 after cloning db1179', diff saved to https://phabricator.wikimedia.org/P15347 and previous config saved to /var/cache/conftool/dbconfig/20210415-071207-root.json |
[production] |
| 06:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 10%: Repool db1166 after cloning db1179', diff saved to https://phabricator.wikimedia.org/P15346 and previous config saved to /var/cache/conftool/dbconfig/20210415-065704-root.json |
[production] |
| 06:33 |
<ryankemper> |
!log T280108 T267927 `data-transfer` to `wdqs1004` was successful; cookbook failed due to a newly introduced minor type error that didn't effect the transfer itself |
[production] |
| 06:32 |
<elukey> |
move hue.wikimedia.org to an-tool1009 (from analytics-tool1001) |
[production] |
| 06:00 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 05:54 |
<Amir1> |
end of cleaning archive of pywikibot-bugs and wikidata-bugs T262773 |
[production] |
| 05:44 |
<Amir1> |
start deleting archive of wikidata-bugs T262773 |
[production] |
| 05:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1166 to clone db1179 T275633', diff saved to https://phabricator.wikimedia.org/P15344 and previous config saved to /var/cache/conftool/dbconfig/20210415-050239-marostegui.json |
[production] |
| 04:14 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:14 |
<ryankemper> |
T280108 T267927 `wdqs2008` (source) caught up on lag, xfering to `wdqs1004`: `sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1004.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927` |
[production] |
| 04:06 |
<ryankemper> |
T280108 T267927 Merged https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/679320, will verify correct behavior of `data-transfer` cookbook |
[production] |
| 01:19 |
<Amir1> |
mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php wikidatawiki --property-id P8671 --new-data-type external-id (T278427) |
[production] |
| 00:50 |
<ejegg> |
updated fundraising CiviCRM from c3342aa4ea to 35a8dd33ba |
[production] |
|
2021-04-14
§
|
| 23:27 |
<ladsgroup@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:679350|Disable legacy javascript global variables in ruwiki (T72470)]] (duration: 01m 16s) |
[production] |
| 21:44 |
<legoktm> |
manually started debmonitor-client.service on ml-serve2004 after 502 Bad gateway error |
[production] |
| 20:55 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on wtp[1037-1039].eqiad.wmnet with reason: reimage |
[production] |
| 20:54 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on wtp[1037-1039].eqiad.wmnet with reason: reimage |
[production] |
| 20:38 |
<mutante> |
wtp1037, wtp1038, wtp1039 - scap pull |
[production] |
| 19:52 |
<dzahn@cumin1001> |
conftool action : set/weight=20; selector: name=mw2395.codfw.wmnet,cluster=jobrunner |
[production] |
| 19:52 |
<dzahn@cumin1001> |
conftool action : set/weight=20; selector: name=mw2394.codfw.wmnet,cluster=jobrunner |
[production] |
| 19:51 |
<dzahn@cumin1001> |
conftool action : set/weight=20; selector: name=mw2410.codfw.wmnet,cluster=videoscaler |
[production] |
| 19:51 |
<dzahn@cumin1001> |
conftool action : set/weight=20; selector: name=mw2411.codfw.wmnet,cluster=videoscaler |
[production] |
| 19:50 |
<cstone> |
civicrm revision changed from ec2a3bcff6 to c3342aa4ea |
[production] |
| 19:50 |
<dzahn@cumin1001> |
conftool action : set/weight=15; selector: name=mw2411.codfw.wmnet,cluster=videoscaler |
[production] |