1051-1100 of 10000 results (83ms)
2019-07-17 ยง
16:20 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:20 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:19 <jijiki> Depool mw2181 - T205240 [production]
16:08 <Urbanecm> Morning SWAT done [production]
16:07 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: Raise zh_classicalwiki requirement for autoconfirmed (T228141) (duration: 00m 55s) [production]
16:07 <cmjohnson1> powering off cloudvirt1014 for rack move T226188 [production]
16:05 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:523686|Enable partial blocks on dewiki]] (T228150) (duration: 00m 54s) [production]
16:01 <jbond42> copy confd package from stretch-wikimedia to buster-wikimedia [production]
15:46 <Urbanecm> Re-syncing patch for T207094 T228284 and wmf.14 [production]
15:37 <Urbanecm> Deployed patch for T207094 T228284 to wmf.13 and wmf.14 [production]
15:15 <fsero> restarting swift-container-sync on ms-be* for getting logging configuration T228196 [production]
15:11 <papaul> shutting down mw2250 for disk replacement [production]
15:10 <gehel@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
15:07 <hashar> upgrading CI Jenkins # T228142 [production]
15:06 <papaul> shutting down ms-be2022 for HW troubleshooting [production]
15:06 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:05 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:03 <jijiki> Depool mw2269 to reboot it - T227548 [production]
15:00 <godog> poweroff ms-be2022 - T227667 [production]
14:55 <moritzm> updated jenkins in thirdparty/ci (stretch) and thirdparty (jessie) to 2.176.2 (T228142) [production]
14:45 <fsero> enabling container-sync logging T228196 [production]
14:41 <otto@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
14:41 <otto@cumin1001> START - Cookbook sre.hosts.decommission [production]
14:35 <moritzm> restart pybal on lvs2002 (codfw primary) T227778 [production]
14:32 <gehel@cumin1001> START - Cookbook sre.postgresql.postgres-init [production]
14:30 <gehel> repool maps1004 - T218097 [production]
14:11 <liw@deploy1001> Synchronized php: group1 wikis to 1.34.0-wmf.14 (duration: 00m 54s) [production]
14:10 <liw@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.34.0-wmf.14 [production]
14:09 <moritzm> restarting pybal on backup LVSes in codfw [production]
14:02 <liw@deploy1001> Synchronized php-1.34.0-wmf.14/extensions/CirrusSearch/includes/Searcher.php: Do not serialize ResultsType instance T228276 (duration: 00m 55s) [production]
13:37 <gehel@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
13:26 <moritzm> disabled puppet on Icinga hosts in preparation of adding the LDAP replicas/codfw to LVS [production]
13:10 <ema> cp-codfw: varnish frontend rolling restarts for 5.1.3-1wm11 upgrades T227672 [production]
13:06 <ema> prometheus servers: remove varnish-upload_$dc_backend.yaml, replaced by ATS equivalent T227668 [production]
12:57 <gehel@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
12:36 <godog> upgrade hp raid firmware on ms-be1 hosts - T141756 [production]
12:15 <Urbanecm> Running foreachwiki extensions/AbuseFilter/maintenance/normalizeThrottleParameters.php in tmux session on mwmaint1002 (T209565) [production]
12:11 <Urbanecm> Ran extensions/AbuseFilter/maintenance/normalizeThrottleParameters.php for cawiki and viwiki (T209565) [production]
11:58 <gehel@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
11:30 <mlitn@deploy1001> Synchronized php-1.34.0-wmf.14/extensions/WikibaseMediaInfo: [WikibaseMediaInfo] Revert "Add Wikidata links to statement UI elements" (duration: 00m 56s) [production]
11:16 <dcausse> reindexing wikidata (elastic@eqiad) T227136 [production]
11:08 <dcausse@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T227136: [cirrus] switch search traffic (except completion) to codfw (duration: 00m 54s) [production]
10:53 <moritzm> re-enabled icinga1001 in meta monitoring [production]
10:41 <godog> install updated linux-image-4.9.0-9-amd64 on ms-be hosts [production]
10:30 <godog> start rolling reboot of ms-be eqiad hosts - T225713 [production]
10:30 <gehel@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
10:23 <moritzm> rebooting icinga1001 for kernel update [production]
10:20 <moritzm> disabled icinga1001 in meta monitoring [production]
10:18 <jmm@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
10:18 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]