4151-4200 of 10000 results (56ms)
2018-07-05 §
05:29 <marostegui> Deploy schema change on s3 primary master (db1075) T191316 T192926 T195193 [production]
04:57 <marostegui> Deploy schema change on s5 codfw host by host T146591 T197891 T196379 [production]
04:50 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Restore db1089 original traffic (duration: 00m 51s) [production]
04:42 <marostegui> Deploy schema change on s7 primary master (db1062) T191316 T192926 T195193 [production]
02:40 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: 370242d13 and 45f4f61c2 (duration: 00m 52s) [production]
02:32 <l10nupdate@deploy1001> ResourceLoader cache refresh completed at Thu Jul 5 02:32:29 UTC 2018 (duration 10m 21s) [production]
02:22 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.10) (duration: 08m 46s) [production]
00:46 <krinkle@deploy1001> Synchronized wmf-config/: Ia3bef874a (duration: 00m 51s) [production]
2018-07-04 §
22:28 <krinkle@deploy1001> Synchronized wmf-config/FeaturedFeedsWMF.php: I004bc9c3e71 (duration: 00m 50s) [production]
22:06 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: I5a6e28814ba6b7 (duration: 00m 51s) [production]
21:53 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Iab176f3d205 (duration: 00m 50s) [production]
21:49 <krinkle@deploy1001> Synchronized wmf-config/: I6f8cfa8f (duration: 00m 51s) [production]
21:43 <krinkle@deploy1001> Synchronized wmf-config/: Ia8deabc5d2625 (duration: 00m 51s) [production]
21:42 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Ia8deabc5d2625 (duration: 00m 50s) [production]
21:36 <krinkle@deploy1001> Synchronized wmf-config/: If7da2a26bbf (duration: 00m 51s) [production]
21:34 <krinkle@deploy1001> Synchronized wmf-config/: I1e78ba4365 (duration: 00m 52s) [production]
21:30 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Ie30aeecbe (duration: 00m 52s) [production]
15:55 <marostegui> Optimize dewiki.logging on s5 codfw master with replication, this will generate lag on s5 codfw - T197459 [production]
15:52 <ema> cp300[3-6]: puppet node clean/deactivate T167376 [production]
15:50 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 for after maintenance (duration: 00m 51s) [production]
14:52 <moritzm> installing libipc-run-perl updates from jessie point release [production]
14:36 <moritzm> installing perl security updates on trusty (Debian already fixed) [production]
14:25 <akosiaris> upgrade kubernetes staging API server to 1.8.14 [production]
14:22 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 for after maintenance (duration: 00m 50s) [production]
14:10 <moritzm> installing file/libmagic security updates on trusty (Debian already fixed) [production]
13:54 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 for after maintenance (duration: 00m 50s) [production]
13:41 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 for after maintenance (duration: 00m 50s) [production]
13:34 <jynus@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2038, db2047 (duration: 02m 56s) [production]
12:56 <marostegui> Stop MySQL and reboot db1089 to upgrade+change it to statement - T197069 [production]
12:46 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1089 for maintenance - T197069 (duration: 02m 57s) [production]
12:39 <ema> cp3034 repooled after hw maintenance T189305 [production]
12:32 <volans> shutting down bast3002 for disk replacement [production]
12:04 <moritzm> installing ruby 1.9 security updates on trusty [production]
11:54 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1077 after alter table (duration: 00m 52s) [production]
11:54 <ema> repool cp3043 after hardware maintenance T179953 [production]
11:30 <ema> shutdown cp3048 and cp3034 (both already depooled) for hardware maintenance T190607 T189305 [production]
11:28 <moritzm> rolling restart of cassandra on restbase hosts in eqiad completed [production]
11:27 <moritzm> resuming rolling restart of cassandra on restbase hosts in eqiad completed [production]
11:17 <mark> cp3043: mdadm /dev/md0 --add /dev/sdc1 (sdc is former cp3048:sdb) [production]
11:09 <mark> cp3043: mdadm /dev/md0 -- fail /dev/sdb1 [production]
11:03 <ema> depool cp3043 (cache_upload) for hardware maintenance T179953 [production]
10:58 <godog> update compiler facts [production]
10:53 <jynus> stop db2038 and db2047 [production]
10:46 <marostegui> Deploy schema change on db1077 with replication, this will generate lag on labs s3 T191316 T192926 T89737 T195193 [production]
10:42 <marostegui> Stop replication on db1077 to drop triggers on db1124:3313 - T192926 [production]
10:39 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1077 for alter table (duration: 00m 50s) [production]
10:28 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1123 after alter table (duration: 00m 50s) [production]
10:01 <moritzm> rolling reboot of sca* for "lazy fpu" kernel updates [production]
09:44 <_joe_> stopping all cronjobs via a puppet run on terbium, T192092 [production]
09:29 <akosiaris> upload kubernetes 1.8.14 to apt.wikimedia.org/stretch-wikimedia/main [production]