4301-4350 of 10000 results (71ms)
2018-01-16 ยง
15:31 <moritzm> rebooting acamar for kernel security update [production]
15:30 <marostegui> Deploy schema change on db1101:3318 - T174569 [production]
15:11 <marostegui> Upgrade mariadb and kernel on db1101 [production]
15:10 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1101:3317 db1101:3318 for schema change, mariadb upgrade and kernel upgrade - T162807 (duration: 01m 12s) [production]
14:59 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1105:3311 - T162807 (duration: 01m 09s) [production]
14:52 <moritzm> rebooting graphite1001 for kernel security update [production]
14:35 <moritzm> rebooting graphite1003 for kernel security update [production]
14:25 <moritzm> powercycling labtestservices2003, stuck in reboot [production]
14:18 <moritzm> powercycling labtestservices2001, stuck in reboot [production]
14:13 <zeljkof> EU SWAT finished [production]
14:12 <elukey> reboot druid100[56] for kernel upgrades [production]
14:11 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:403571|Revert "Restrict sending mails to new users" config change (T184470)]] (duration: 01m 13s) [production]
14:09 <godog> bootstrap cassandra-b on restbase1018 [production]
14:01 <moritzm> rebooting labtest* hosts for kernel security update [production]
13:56 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=druid1004.*.wmnet [production]
13:54 <moritzm> rebooting graphite1002 for kernel security update [production]
13:52 <elukey> reboot druid1004 for kernel upgrades [production]
13:46 <elukey@puppetmaster1001> conftool action : set/pooled=no; selector: name=druid1004.*.wmnet [production]
13:46 <elukey@puppetmaster1001> conftool action : set/pooled=no; selector: name=druid1004*.wmnet [production]
13:44 <moritzm> rebooting graphite2002 for kernel security update [production]
13:31 <oblivian@neodymium> conftool action : set/weight=25; selector: cluster=api_appserver,name=mw134[3-8[B]\.eqiad\.wmnet [production]
13:28 <moritzm> rebooting graphite2001 for kernel security update [production]
13:20 <oblivian@neodymium> conftool action : set/pooled=yes; selector: cluster=api_appserver,name=mw134[3-7]\.eqiad\.wmnet [production]
12:56 <elukey> reboot kafka100[23] for kernel upgrades [production]
11:59 <ariel@tin> Finished deploy [dumps/dumps@c165ca0]: enable 7z prefetch files for page content dumps (duration: 00m 04s) [production]
11:59 <ariel@tin> Started deploy [dumps/dumps@c165ca0]: enable 7z prefetch files for page content dumps [production]
11:51 <moritzm> rebooting mc2* hosts for kernel security update [production]
11:27 <elukey> reboot kafka1001 for kernel upgrades [production]
11:12 <moritzm> reboot maerlant for kernel security update [production]
11:08 <moritzm> uploaded HHVM 3.18.7 for stretch-wikimedia to apt.wikimedia.org [production]
10:59 <godog> roll-restart swift object server - T167400 [production]
10:57 <moritzm> reboot nescio for kernel security update [production]
10:13 <godog> start cassandra-a on restbase1018 - T184100 [production]
09:56 <moritzm> upgrading canary app servers to HHVM 3.18.7 [production]
09:50 <marostegui> Stop replication in sync db1089 - db1105:3311 - T162807 [production]
09:43 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1105:3311 - T162807 (duration: 01m 12s) [production]
09:38 <_joe_> started refreshLinks additional jobs for commonswiki,ruwiki [production]
09:30 <oblivian@neodymium> conftool action : set/weight=10; selector: cluster=api_appserver,name=mw13(39|4[12]).eqiad.wmnet [production]
09:24 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1099:3311 - T162807 (duration: 01m 12s) [production]
09:24 <oblivian@neodymium> conftool action : set/pooled=yes:weight=1; selector: cluster=api_appserver,name=mw13(39|4[12]).eqiad.wmnet [production]
09:16 <moritzm> installing libxml2 security updates on mw* servers (so that it gets picked up along the HHVM 3.18.7 rollout) [production]
09:12 <moritzm> installing krb5 security updates (we're just using rev deps) [production]
09:08 <jynus> upgrade and reboot db1031 after switchover [production]
08:49 <moritzm> rearmed key holder on sarin [production]
08:45 <moritzm> rebooting sarin for kernel security update [production]
08:38 <marostegui> Stop replication in sync db1089 - db1099:3311 - T162807 [production]
08:36 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1066, depool db1099:3311 - T162807 (duration: 01m 12s) [production]
08:28 <jynus> master x1 eqiad failover has finished [production]
08:19 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Promote db1055 as the new x1 master (duration: 00m 49s) [production]
08:17 <jynus> setting db1031 (x1 master) as read only [production]