|
2018-01-02
ยง
|
| 16:51 |
<herron> |
restarted exim and spamd services on fermium, mx1001 and mx2001 for openssl update |
[production] |
| 16:48 |
<elukey@puppetmaster1001> |
conftool action : set/weight=30; selector: name=mw13(29|3[0-3]).*.eqiad.wmnet |
[production] |
| 16:24 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1029 (duration: 00m 51s) |
[production] |
| 15:55 |
<moritzm> |
installing openssl updates on restbase* hosts |
[production] |
| 15:53 |
<elukey@puppetmaster1001> |
conftool action : set/weight=20; selector: name=mw13(29|3[0-3]).*.eqiad.wmnet |
[production] |
| 15:33 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw1335.*.eqiad.wmnet |
[production] |
| 15:29 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1055 & db1056 x1 weight (duration: 00m 50s) |
[production] |
| 15:16 |
<akosiaris> |
boot ganeti1008 with older 4.4 kernel and migrate multiple VMs to it. T181121 |
[production] |
| 15:05 |
<zeljkof> |
EU SWAT finished |
[production] |
| 15:04 |
<zfilipin@tin> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:400579|Set watchcreations preference to true by default on Commons (T178750)]] (duration: 00m 51s) |
[production] |
| 15:03 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:400579|Set watchcreations preference to true by default on Commons (T178750)]] (duration: 00m 51s) |
[production] |
| 14:54 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:400096|Enable mapframe on lvwiki (T183661)]] (duration: 00m 51s) |
[production] |
| 14:42 |
<zfilipin@tin> |
Synchronized static/images/project-logos/: SWAT: [[gerrit:401509|Revert "Switch Wikipedias from $wgLogoHD to direct using of a SVG" (T178942)]] (duration: 00m 51s) |
[production] |
| 14:41 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:401509|Revert "Switch Wikipedias from $wgLogoHD to direct using of a SVG" (T178942)]] (duration: 00m 51s) |
[production] |
| 14:33 |
<zfilipin@tin> |
scap failed: average error rate on 6/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) |
[production] |
| 14:32 |
<zfilipin@tin> |
Synchronized static/images/project-logos/: SWAT: [[gerrit:399805|Switch Wikipedias from $wgLogoHD to direct using of a SVG (T178942)]] (duration: 01m 59s) |
[production] |
| 14:17 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:400409|Add suppressredirect to autoreview/editor at ruwikt (T183719)]] (duration: 00m 51s) |
[production] |
| 14:12 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:400093|Create rollbacker user group for ruwiktionary (T183655)]] (duration: 00m 52s) |
[production] |
| 14:07 |
<foks> |
removed 2FA for Martin_Urbanec |
[production] |
| 14:00 |
<moritzm> |
installing further openssl updates |
[production] |
| 13:49 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw1333.*.eqiad.wmnet |
[production] |
| 13:48 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw1332.*.eqiad.wmnet |
[production] |
| 13:47 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw1331.*.eqiad.wmnet |
[production] |
| 13:45 |
<marostegui> |
Deploy alter table db1098:3317 - T174569 |
[production] |
| 13:45 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw1330.*.eqiad.wmnet |
[production] |
| 13:44 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1098:3317 - T174569 (duration: 00m 51s) |
[production] |
| 13:42 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw1329.*.eqiad.wmnet |
[production] |
| 13:41 |
<elukey> |
enable live traffic for new appservers mw1329->mw1333 (T165519) |
[production] |
| 13:00 |
<moritzm> |
installing openssl updates on remaining mw* hosts in eqiad |
[production] |
| 12:25 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db1055 & db1056 as x1 replicas (duration: 00m 51s) |
[production] |
| 12:24 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1055 & db1056 as x1 replicas (2nd try) (duration: 00m 51s) |
[production] |
| 12:21 |
<akosiaris> |
empty ganeti1008 for kernel downgrade. T181121 |
[production] |
| 12:11 |
<jynus> |
add missing mysql grants to db1055 and db1056 |
[production] |
| 11:42 |
<moritzm> |
installing ncurses security updates |
[production] |
| 11:39 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Revert: Repool db1055 & db1056 as x1 replicas (duration: 00m 51s) |
[production] |
| 11:32 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1055 & db1056 as x1 replicas (duration: 00m 50s) |
[production] |
| 11:31 |
<mobrovac@tin> |
Finished deploy [citoid/deploy@ee0bdf4]: Update to service template v0.5.4 - T151394 (duration: 04m 19s) |
[production] |
| 11:30 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db1055 & db1056 as x1 replicas (duration: 00m 50s) |
[production] |
| 11:27 |
<mobrovac@tin> |
Started deploy [citoid/deploy@ee0bdf4]: Update to service template v0.5.4 - T151394 |
[production] |
| 09:45 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: cluster=api_appserver,name=mw1(1.*|200).eqiad.wmnet |
[production] |
| 09:44 |
<_joe_> |
setting api_appservers in the mw1180-1200 range to pooled=inactive, T183895 |
[production] |
| 09:43 |
<oblivian@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: cluster=appserver,name=mw1(1.*|200).eqiad.wmnet |
[production] |
| 09:37 |
<_joe_> |
setting appservers in the mw1180-1200 range to pooled=inactive, T183895 |
[production] |
| 09:28 |
<godog> |
reboot ms-be1033 - T183724 |
[production] |
| 08:52 |
<_joe_> |
restarting also mw1226-8, mw1223, mw1201,mw1203, mw1205-7 |
[production] |
| 08:36 |
<_joe_> |
likewise for mw1285,mw1235,mw1232 |
[production] |
| 08:29 |
<_joe_> |
restarting hhvm on mw1280,1282 for the same reasons |
[production] |
| 08:26 |
<_joe_> |
restarting hhvm on mw1317, multiple threads stuck in HPHP::jit::enterTCImpl |
[production] |
| 08:23 |
<elukey> |
restart druid coordinators on druid* to pick up new jvm settings |
[production] |
| 08:19 |
<_joe_> |
restarting hhvm on mw1313, concurrency HPHP::VariableUnserializer::unserializeVariant |
[production] |