3501-3550 of 10000 results (52ms)
2017-12-18 §
13:58 <jynus> starting one-time backup of eventlogging database on db1107:/srv/backups T183123 [production]
13:29 <marostegui> Stop replicaiton in sync on db1109 and db2045 - T161294 [production]
13:25 <jmm@puppetmaster1001> conftool action : set/pooled=yes; selector: mw1307.eqiad.wmnet [production]
13:25 <jmm@puppetmaster1001> conftool action : set/pooled=yes; selector: mw1313.eqiad.wmnet [production]
13:21 <jmm@puppetmaster1001> conftool action : set/pooled=no; selector: mw1313.eqiad.wmnet [production]
13:20 <jmm@puppetmaster1001> conftool action : set/pooled=yes; selector: mw1260.eqiad.wmnet [production]
13:20 <jmm@puppetmaster1001> conftool action : set/pooled=no; selector: mw1260.eqiad.wmnet [production]
13:18 <marostegui> Stop replication in sync on db1100 and db2052 - T161294 [production]
13:10 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1106 - T161294 (duration: 03m 06s) [production]
13:00 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1106 - T161294 (duration: 03m 03s) [production]
12:58 <marostegui> Stop replication in sync on db1106 and db1100 - T161294 [production]
12:35 <marostegui> Deploy schema change on db1099:331 and db1067 - T174569 [production]
12:34 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1099:3311 - T174569 (duration: 03m 05s) [production]
12:25 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1066 - T174569 (duration: 03m 06s) [production]
12:20 <akosiaris> build scap 3.7.4-3 and upload to jessie-wikimedia, stretch-wikimedia, trusty-wikimedia. T183046, T182347 [production]
11:37 <hashar> restarting Jenkins CI to upgrade the monitoring plugin [production]
11:24 <_joe_> ran cleanup script on scb* T180384 [production]
11:20 <mobrovac> stopping the trending edits service - T180384 [production]
11:18 <moritzm> reimaging mw1307 (video scaler) to stretch [production]
11:08 <_joe_> rolling restart of pybal on the low-traffic balancers [production]
11:04 <_joe_> disabled notifications for trendingedits.svc T180384 [production]
10:46 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1109 - T161294 (duration: 00m 56s) [production]
10:30 <marostegui> Stop replication on db1109 and db2045 in sync - T161294 [production]
10:30 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1109 - T161294 (duration: 00m 56s) [production]
09:47 <marostegui> Deploy schema change on db1066 - T174569 [production]
09:47 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1066 - T174569 (duration: 00m 56s) [production]
09:43 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1073 - T174569 (duration: 00m 57s) [production]
09:40 <moritzm> installing openssl security updates [production]
09:39 <marostegui> Deploy schema change on db1055 (already depooled) - T174569 [production]
09:28 <gehel> removing initial import datafiles from maps[12]001 [production]
08:58 <marostegui> Stop replication in sync on db1100 and db2052 - T161294 [production]
08:57 <elukey> rolling restart of the Yarn nodemanagers (hadoop) on analytics10[456]* to pick up new settings - T182276 [production]
08:30 <Jamesofur> insert decryption key for 2017 Arb elections [production]
08:08 <volans> powercycling ganeti1005 [production]
06:57 <_joe_> reeanbling puppet across servers with scap [production]
06:44 <marostegui> Defragment s2 databases on db1102 - T172169 [production]
06:43 <_joe_> restarted hhvm on mw1283, still the same kind of lockups [production]
06:33 <marostegui> Deploy schema change on db1073 - T174569 [production]
06:32 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1073 - T174569 (duration: 00m 57s) [production]
06:28 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1106 - T161294 (duration: 00m 56s) [production]
06:17 <marostegui> Stop replication in sync on db1106 and db1100 - T161294 [production]
06:17 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1106 - T161294 (duration: 00m 57s) [production]
03:02 <eileen> re-enable multiqueue_consumer process-control config revision is 1ae3778278 [production]
03:00 <eileen> civicrm revision changed from 43d3f4d739 to e0ee2d189c [production]
02:32 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.12) (duration: 05m 57s) [production]
01:17 <cwd> disabled fredge multiqueue consumer [production]
00:49 <eileen> civicrm revision changed from 798e24671b to 43d3f4d739 [production]
2017-12-16 §
17:27 <no_justification> gerrit: Back, might see a few transient puppet failures if git pulls happened during the d/t, but should all recover [production]
17:21 <no_justification> gerrit: halting service momentarily for account reindexing [production]
15:00 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1100 (duration: 00m 58s) [production]