1701-1750 of 10000 results (68ms)
2020-03-20 §
21:06 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
21:04 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
21:01 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
20:59 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
20:59 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
20:59 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
20:57 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
20:56 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
20:55 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
20:53 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
20:41 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw124[4-9].eqiad.wmnet [production]
20:40 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:40 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:40 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:40 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:40 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:39 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:37 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw123[0-1].eqiad.wmnet [production]
20:32 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw122[7-9].eqiad.wmnet [production]
20:18 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw124[4-9].eqiad.wmnet [production]
20:18 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw123[0-1].eqiad.wmnet [production]
20:18 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw122[7-9].eqiad.wmnet [production]
15:44 <hashar@deploy1001> Synchronized php-1.35.0-wmf.24/includes/ActorMigration.php: Avoid upsert() log warning spam in ActorMigration due to unique key array format - T248147 (duration: 01m 01s) [production]
13:34 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
13:33 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
13:33 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
12:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Decrease db1087, vslow host weight in main, given that the CPU across s8 is now doing a lot better', diff saved to https://phabricator.wikimedia.org/P10741 and previous config saved to /var/cache/conftool/dbconfig/20200320-121628-marostegui.json [production]
11:52 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
11:10 <elukey> upload oozie 4.3.0-2 packages to thirdparty/bigtop14 on wikimedia-stretch [production]
10:56 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
10:56 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
10:34 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
10:29 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
10:13 <dcausse> repooling wdqs1006 [production]
09:28 <moritzm> rolling restart of FPM on mw1261-mw1265 for freetype update [production]
08:59 <moritzm> installing freetype bugfix updates from stretch point release [production]
08:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool es1017', diff saved to https://phabricator.wikimedia.org/P10739 and previous config saved to /var/cache/conftool/dbconfig/20200320-084730-marostegui.json [production]
08:33 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool es1017', diff saved to https://phabricator.wikimedia.org/P10738 and previous config saved to /var/cache/conftool/dbconfig/20200320-083334-marostegui.json [production]
07:59 <XioNoX> reorder LVS BGP neighbors and add descriptions - https://gerrit.wikimedia.org/r/576320 [production]
07:48 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool es1017', diff saved to https://phabricator.wikimedia.org/P10737 and previous config saved to /var/cache/conftool/dbconfig/20200320-074816-marostegui.json [production]
07:46 <elukey> upload hadoop_2.8.5-2 (and related debs) to thirdparty/bigtop14 on wikimedia-stretch (manually rebuilt via docker after patch backports from upstream) [production]
07:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool es1017', diff saved to https://phabricator.wikimedia.org/P10736 and previous config saved to /var/cache/conftool/dbconfig/20200320-073205-marostegui.json [production]
07:26 <marostegui> Restart mysql on es1017 for upgrade - T239791 [production]
07:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es1017 for update T239791', diff saved to https://phabricator.wikimedia.org/P10735 and previous config saved to /var/cache/conftool/dbconfig/20200320-070945-marostegui.json [production]
07:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Promote es1014 to es3 master, this is a NOOP T239791', diff saved to https://phabricator.wikimedia.org/P10734 and previous config saved to /var/cache/conftool/dbconfig/20200320-070922-marostegui.json [production]
2020-03-19 §
22:15 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@794f099]: Update mobileapps to 99869f45 (duration: 05m 13s) [production]
22:10 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@794f099]: Update mobileapps to 99869f45 [production]
19:14 <hashar@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.24 [production]
18:30 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.24/extensions/Wikibase/lib/includes/Store/ByIdDispatchingEntityInfoBuilder.php: [[gerrit:581674|Fix 'max' to Int32EntityId::MAX conversion (T247985)]], part II (duration: 01m 07s) [production]
18:24 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.24/extensions/Wikibase/data-access/src/SingleEntitySourceServices.php: [[gerrit:581674|Fix 'max' to Int32EntityId::MAX conversion (T247985)]], part I (duration: 01m 08s) [production]