1801-1850 of 10000 results (59ms)
2019-10-24 ยง
13:17 <ema> set ats-be weights on new esams upload nodes T233242 [production]
13:06 <liw@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.3 [production]
12:56 <effie> purge hhvm hhvm-luasandbox hhvm-tidy hhvm-wikidiff2 hhvm-dbg from mw* canaries - T229792 [production]
12:42 <ema@puppetmaster1001> conftool action : set/weight=100; selector: name=cp3060.esams.wmnet,service=varnish-be [production]
12:33 <effie> Stopping puppet on all hosts including the hhvm class (C:hhvm) - 544864 - T229792 [production]
12:25 <ema> cp3060: powercycle -- NMI watchdog: BUG: soft lockup - CPU#18 stuck for 22s! [charon:1226] T233242 [production]
12:14 <bblack> depool esams in geodns [production]
12:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2092 after analyze table', diff saved to https://phabricator.wikimedia.org/P9468 and previous config saved to /var/cache/conftool/dbconfig/20191024-120812-marostegui.json [production]
12:06 <XioNoX> shutdown cr1-esams - cr2-knams link [production]
12:00 <XioNoX> shutdown transit BGP sessions on cr2-knams [production]
11:40 <Urbanecm> EU SWAT done [production]
11:35 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 3a5cb68: Permission changes of move-rootuserpages assignment at commonswiki (T236359) (duration: 01m 00s) [production]
11:33 <ema@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
11:31 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:30 <Urbanecm> Run mwscript namespaceDupes.php --wiki=commonswiki --add-prefix=FIXME --fix (T236352) [production]
11:28 <vgutierrez@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
11:26 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:26 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: e079956: Add CAT as alias for NS_CATEGORY at commonswiki (T236352) (duration: 01m 00s) [production]
11:22 <urbanecm@deploy1001> Synchronized dblists/commonsuploads.dblist: SWAT: 2d66deb: Restrict uploads on azwiki (T236307) (duration: 01m 03s) [production]
11:15 <mlitn@deploy1001> Synchronized php-1.35.0-wmf.3/extensions/WikibaseMediaInfo: Also use custom PrefetchingTermLookup in SingleEntitySourceServices (duration: 01m 01s) [production]
11:13 <mlitn@deploy1001> Synchronized php-1.35.0-wmf.3/extensions/Wikibase: Allow defining entity-type-specific PrefetchingTermLookup (duration: 01m 06s) [production]
11:08 <aborrero@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
11:08 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:08 <aborrero@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
11:08 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:00 <vgutierrez@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
10:58 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:56 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:56 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:56 <aborrero@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
10:56 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:56 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:56 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:56 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:56 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:56 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:56 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:56 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:56 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:55 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:55 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:55 <ema@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
10:52 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Adjust s6 weights for db1093 and db1085', diff saved to https://phabricator.wikimedia.org/P9466 and previous config saved to /var/cache/conftool/dbconfig/20191024-101810-marostegui.json [production]
09:59 <hashar> Converting CI jobs to use the new PostBuildScript plugin config | https://gerrit.wikimedia.org/r/#/c/integration/config/+/544907/ | T188398 [production]
09:57 <hashar> Restarting CI Jenkins [production]
09:35 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:33 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:14 <vgutierrez@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
09:12 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]