701-750 of 10000 results (46ms)
2018-01-02 ยง
16:51 <herron> restarted exim and spamd services on fermium, mx1001 and mx2001 for openssl update [production]
16:48 <elukey@puppetmaster1001> conftool action : set/weight=30; selector: name=mw13(29|3[0-3]).*.eqiad.wmnet [production]
16:24 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1029 (duration: 00m 51s) [production]
15:55 <moritzm> installing openssl updates on restbase* hosts [production]
15:53 <elukey@puppetmaster1001> conftool action : set/weight=20; selector: name=mw13(29|3[0-3]).*.eqiad.wmnet [production]
15:33 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=mw1335.*.eqiad.wmnet [production]
15:29 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Increase db1055 & db1056 x1 weight (duration: 00m 50s) [production]
15:16 <akosiaris> boot ganeti1008 with older 4.4 kernel and migrate multiple VMs to it. T181121 [production]
15:05 <zeljkof> EU SWAT finished [production]
15:04 <zfilipin@tin> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:400579|Set watchcreations preference to true by default on Commons (T178750)]] (duration: 00m 51s) [production]
15:03 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:400579|Set watchcreations preference to true by default on Commons (T178750)]] (duration: 00m 51s) [production]
14:54 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:400096|Enable mapframe on lvwiki (T183661)]] (duration: 00m 51s) [production]
14:42 <zfilipin@tin> Synchronized static/images/project-logos/: SWAT: [[gerrit:401509|Revert "Switch Wikipedias from $wgLogoHD to direct using of a SVG" (T178942)]] (duration: 00m 51s) [production]
14:41 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:401509|Revert "Switch Wikipedias from $wgLogoHD to direct using of a SVG" (T178942)]] (duration: 00m 51s) [production]
14:33 <zfilipin@tin> scap failed: average error rate on 6/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) [production]
14:32 <zfilipin@tin> Synchronized static/images/project-logos/: SWAT: [[gerrit:399805|Switch Wikipedias from $wgLogoHD to direct using of a SVG (T178942)]] (duration: 01m 59s) [production]
14:17 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:400409|Add suppressredirect to autoreview/editor at ruwikt (T183719)]] (duration: 00m 51s) [production]
14:12 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:400093|Create rollbacker user group for ruwiktionary (T183655)]] (duration: 00m 52s) [production]
14:07 <foks> removed 2FA for Martin_Urbanec [production]
14:00 <moritzm> installing further openssl updates [production]
13:49 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=mw1333.*.eqiad.wmnet [production]
13:48 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=mw1332.*.eqiad.wmnet [production]
13:47 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=mw1331.*.eqiad.wmnet [production]
13:45 <marostegui> Deploy alter table db1098:3317 - T174569 [production]
13:45 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=mw1330.*.eqiad.wmnet [production]
13:44 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1098:3317 - T174569 (duration: 00m 51s) [production]
13:42 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=mw1329.*.eqiad.wmnet [production]
13:41 <elukey> enable live traffic for new appservers mw1329->mw1333 (T165519) [production]
13:00 <moritzm> installing openssl updates on remaining mw* hosts in eqiad [production]
12:25 <jynus@tin> Synchronized wmf-config/db-codfw.php: Repool db1055 & db1056 as x1 replicas (duration: 00m 51s) [production]
12:24 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1055 & db1056 as x1 replicas (2nd try) (duration: 00m 51s) [production]
12:21 <akosiaris> empty ganeti1008 for kernel downgrade. T181121 [production]
12:11 <jynus> add missing mysql grants to db1055 and db1056 [production]
11:42 <moritzm> installing ncurses security updates [production]
11:39 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Revert: Repool db1055 & db1056 as x1 replicas (duration: 00m 51s) [production]
11:32 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1055 & db1056 as x1 replicas (duration: 00m 50s) [production]
11:31 <mobrovac@tin> Finished deploy [citoid/deploy@ee0bdf4]: Update to service template v0.5.4 - T151394 (duration: 04m 19s) [production]
11:30 <jynus@tin> Synchronized wmf-config/db-codfw.php: Repool db1055 & db1056 as x1 replicas (duration: 00m 50s) [production]
11:27 <mobrovac@tin> Started deploy [citoid/deploy@ee0bdf4]: Update to service template v0.5.4 - T151394 [production]
09:45 <oblivian@puppetmaster1001> conftool action : set/pooled=inactive; selector: cluster=api_appserver,name=mw1(1.*|200).eqiad.wmnet [production]
09:44 <_joe_> setting api_appservers in the mw1180-1200 range to pooled=inactive, T183895 [production]
09:43 <oblivian@puppetmaster1001> conftool action : set/pooled=inactive; selector: cluster=appserver,name=mw1(1.*|200).eqiad.wmnet [production]
09:37 <_joe_> setting appservers in the mw1180-1200 range to pooled=inactive, T183895 [production]
09:28 <godog> reboot ms-be1033 - T183724 [production]
08:52 <_joe_> restarting also mw1226-8, mw1223, mw1201,mw1203, mw1205-7 [production]
08:36 <_joe_> likewise for mw1285,mw1235,mw1232 [production]
08:29 <_joe_> restarting hhvm on mw1280,1282 for the same reasons [production]
08:26 <_joe_> restarting hhvm on mw1317, multiple threads stuck in HPHP::jit::enterTCImpl [production]
08:23 <elukey> restart druid coordinators on druid* to pick up new jvm settings [production]
08:19 <_joe_> restarting hhvm on mw1313, concurrency HPHP::VariableUnserializer::unserializeVariant [production]