6201-6250 of 10000 results (86ms)
2019-10-24 §
04:59 <marostegui@cumin1001> dbctl commit (dc=all): 'More traffic to db1097:3315 after compression', diff saved to https://phabricator.wikimedia.org/P9462 and previous config saved to /var/cache/conftool/dbconfig/20191024-045954-marostegui.json [production]
04:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove db1089 from special slaves group and leave it with its original pooling options T223151', diff saved to https://phabricator.wikimedia.org/P9461 and previous config saved to /var/cache/conftool/dbconfig/20191024-045924-marostegui.json [production]
04:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1097:3315 after compression', diff saved to https://phabricator.wikimedia.org/P9460 and previous config saved to /var/cache/conftool/dbconfig/20191024-045544-marostegui.json [production]
04:48 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
04:48 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission [production]
03:55 <shdubsh> temporarily turn down accept delay on fermium - T235983 [production]
00:03 <mutante> restarting gerrit to increase heap_size from 20G to 32G (T225166 T222391) [production]
2019-10-23 §
22:55 <brennen@deploy1001> Synchronized php-1.35.0-wmf.3/extensions/AbuseFilter: SWAT: [[gerrit:545620|Unbreak filter edit form (T236286)]] (duration: 01m 05s) [production]
22:20 <twentyafterfour@deploy1001> Finished deploy [phabricator/deployment@e4e2b22]: deploy to phab1001 (currently a warm spare server) (duration: 00m 21s) [production]
22:20 <twentyafterfour@deploy1001> Started deploy [phabricator/deployment@e4e2b22]: deploy to phab1001 (currently a warm spare server) [production]
22:20 <twentyafterfour@deploy1001> Finished deploy [phabricator/deployment@e4e2b22]: deploy to phab1001 (currently a warm spare server) (duration: 00m 05s) [production]
22:19 <twentyafterfour@deploy1001> Started deploy [phabricator/deployment@e4e2b22]: deploy to phab1001 (currently a warm spare server) [production]
22:15 <twentyafterfour@deploy1001> Finished deploy [phabricator/deployment@e4e2b22]: deploy to phab1001 (currently a warm spare server) (duration: 01m 10s) [production]
22:14 <twentyafterfour@deploy1001> Started deploy [phabricator/deployment@e4e2b22]: deploy to phab1001 (currently a warm spare server) [production]
22:00 <twentyafterfour@deploy1001> Finished deploy [phabricator/deployment@e4e2b22]: deploy to phab1001 (currently a warm spare server) (duration: 00m 21s) [production]
22:00 <twentyafterfour@deploy1001> Started deploy [phabricator/deployment@e4e2b22]: deploy to phab1001 (currently a warm spare server) [production]
21:32 <mutante> webperf1002/2002 - starting bacula-fd service that is failed after initial puppet run turning them into backup::hosts [production]
21:14 <ejegg> updated Fundraising python tools from b3c7453be2 to ffc7bf764b [production]
20:37 <shdubsh> restart nagios-nrpe-server on stat1007 [production]
18:56 <milimetric@deploy1001> Finished deploy [analytics/refinery@3aaabf6]: Minor: fix two scripts (duration: 07m 53s) [production]
18:49 <milimetric@deploy1001> Started deploy [analytics/refinery@3aaabf6]: Minor: fix two scripts [production]
18:29 <mforns@deploy1001> Finished deploy [analytics/refinery@1110d59]: deploying refinery up to 1110d59c3983bcff4986bce1baf885f05ee06ba5 (duration: 06m 40s) [production]
18:22 <mforns@deploy1001> Started deploy [analytics/refinery@1110d59]: deploying refinery up to 1110d59c3983bcff4986bce1baf885f05ee06ba5 [production]
17:31 <akosiaris> restart varnish-be on cp1089 as a response to HTTP availability alerts. High mailbox lag [production]
17:25 <akosiaris> restart varnish-be on cp1081 as a response to HTTP availability alerts [production]
15:55 <_joe_> restarting pybal on lvs2006, then 2003 for picking up parsoid-php [production]
15:32 <marostegui> Enable slow query log 1/20 on db1089 (enwiki) T223151 [production]
14:40 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
14:39 <ema@cumin1001> START - Cookbook sre.hosts.decommission [production]
14:38 <ema@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
14:37 <ema@cumin1001> START - Cookbook sre.hosts.decommission [production]
14:36 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
14:35 <ema@cumin1001> START - Cookbook sre.hosts.decommission [production]
14:19 <bblack> repooling esams [production]
14:00 <hashar> Restarting CI Jenkins [production]
13:57 <_joe_> manually changing the symlinked deployed version of parsoid on wtp1025 T236275 [production]
13:35 <XioNoX> migrate esams mgmt to new mgmt router [production]
13:34 <effie> disable puppet on mwdebug1002 - T214734 [production]
13:13 <ssastry@deploy1001> Finished deploy [parsoid/deploy@451db1e]: Updating Parsoid to 5521ea74; Dummy Parsoid deploy to debug Parsoid/PHP deployment issues (duration: 08m 44s) [production]
13:07 <liw@deploy1001> Synchronized php: group1 wikis to 1.35.0-wmf.3 (duration: 01m 00s) [production]
13:05 <liw@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.3 [production]
13:04 <ssastry@deploy1001> Started deploy [parsoid/deploy@451db1e]: Updating Parsoid to 5521ea74; Dummy Parsoid deploy to debug Parsoid/PHP deployment issues [production]
12:37 <effie> Depool mwdebug1002 - T214734 [production]
12:31 <vgutierrez> restarting ats-tls on cache text nodes - T233274 [production]
12:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove db1130 from the special slaves group on s5 and leave it back with its original pooling options T223151', diff saved to https://phabricator.wikimedia.org/P9454 and previous config saved to /var/cache/conftool/dbconfig/20191023-122708-marostegui.json [production]
11:26 <XioNoX> powering down cr1-esams [production]
11:24 <Urbanecm> EU SWAT done [production]
11:22 <urbanecm@deploy1001> Synchronized wmf-config/InterwikiSortOrders.php: SWAT: e21054e: Add Balinese to interwiki sort orders (T234768) (duration: 01m 01s) [production]
11:18 <Urbanecm> mwscript updateArticleCount.php --wiki=frwikiquote --update (T236212) [production]
11:16 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 0889da0: Add custom Minerva wordmark for Hebrew wikivoyage (2/2; T234278) (duration: 01m 01s) [production]