|
2018-07-05
§
|
| 05:50 |
<marostegui> |
Deploy schema change on dbstore1002:s5 T146591 T197891 T196379 |
[production] |
| 05:29 |
<marostegui> |
Deploy schema change on s3 primary master (db1075) T191316 T192926 T195193 |
[production] |
| 04:57 |
<marostegui> |
Deploy schema change on s5 codfw host by host T146591 T197891 T196379 |
[production] |
| 04:50 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Restore db1089 original traffic (duration: 00m 51s) |
[production] |
| 04:42 |
<marostegui> |
Deploy schema change on s7 primary master (db1062) T191316 T192926 T195193 |
[production] |
| 02:40 |
<krinkle@deploy1001> |
Synchronized wmf-config/CommonSettings.php: 370242d13 and 45f4f61c2 (duration: 00m 52s) |
[production] |
| 02:32 |
<l10nupdate@deploy1001> |
ResourceLoader cache refresh completed at Thu Jul 5 02:32:29 UTC 2018 (duration 10m 21s) |
[production] |
| 02:22 |
<l10nupdate@deploy1001> |
scap sync-l10n completed (1.32.0-wmf.10) (duration: 08m 46s) |
[production] |
| 00:46 |
<krinkle@deploy1001> |
Synchronized wmf-config/: Ia3bef874a (duration: 00m 51s) |
[production] |
|
2018-07-04
§
|
| 22:28 |
<krinkle@deploy1001> |
Synchronized wmf-config/FeaturedFeedsWMF.php: I004bc9c3e71 (duration: 00m 50s) |
[production] |
| 22:06 |
<krinkle@deploy1001> |
Synchronized wmf-config/CommonSettings.php: I5a6e28814ba6b7 (duration: 00m 51s) |
[production] |
| 21:53 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Iab176f3d205 (duration: 00m 50s) |
[production] |
| 21:49 |
<krinkle@deploy1001> |
Synchronized wmf-config/: I6f8cfa8f (duration: 00m 51s) |
[production] |
| 21:43 |
<krinkle@deploy1001> |
Synchronized wmf-config/: Ia8deabc5d2625 (duration: 00m 51s) |
[production] |
| 21:42 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Ia8deabc5d2625 (duration: 00m 50s) |
[production] |
| 21:36 |
<krinkle@deploy1001> |
Synchronized wmf-config/: If7da2a26bbf (duration: 00m 51s) |
[production] |
| 21:34 |
<krinkle@deploy1001> |
Synchronized wmf-config/: I1e78ba4365 (duration: 00m 52s) |
[production] |
| 21:30 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Ie30aeecbe (duration: 00m 52s) |
[production] |
| 15:55 |
<marostegui> |
Optimize dewiki.logging on s5 codfw master with replication, this will generate lag on s5 codfw - T197459 |
[production] |
| 15:52 |
<ema> |
cp300[3-6]: puppet node clean/deactivate T167376 |
[production] |
| 15:50 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 for after maintenance (duration: 00m 51s) |
[production] |
| 14:52 |
<moritzm> |
installing libipc-run-perl updates from jessie point release |
[production] |
| 14:36 |
<moritzm> |
installing perl security updates on trusty (Debian already fixed) |
[production] |
| 14:25 |
<akosiaris> |
upgrade kubernetes staging API server to 1.8.14 |
[production] |
| 14:22 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 for after maintenance (duration: 00m 50s) |
[production] |
| 14:10 |
<moritzm> |
installing file/libmagic security updates on trusty (Debian already fixed) |
[production] |
| 13:54 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 for after maintenance (duration: 00m 50s) |
[production] |
| 13:41 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1089 for after maintenance (duration: 00m 50s) |
[production] |
| 13:34 |
<jynus@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2038, db2047 (duration: 02m 56s) |
[production] |
| 12:56 |
<marostegui> |
Stop MySQL and reboot db1089 to upgrade+change it to statement - T197069 |
[production] |
| 12:46 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1089 for maintenance - T197069 (duration: 02m 57s) |
[production] |
| 12:39 |
<ema> |
cp3034 repooled after hw maintenance T189305 |
[production] |
| 12:32 |
<volans> |
shutting down bast3002 for disk replacement |
[production] |
| 12:04 |
<moritzm> |
installing ruby 1.9 security updates on trusty |
[production] |
| 11:54 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1077 after alter table (duration: 00m 52s) |
[production] |
| 11:54 |
<ema> |
repool cp3043 after hardware maintenance T179953 |
[production] |
| 11:30 |
<ema> |
shutdown cp3048 and cp3034 (both already depooled) for hardware maintenance T190607 T189305 |
[production] |
| 11:28 |
<moritzm> |
rolling restart of cassandra on restbase hosts in eqiad completed |
[production] |
| 11:27 |
<moritzm> |
resuming rolling restart of cassandra on restbase hosts in eqiad completed |
[production] |
| 11:17 |
<mark> |
cp3043: mdadm /dev/md0 --add /dev/sdc1 (sdc is former cp3048:sdb) |
[production] |
| 11:09 |
<mark> |
cp3043: mdadm /dev/md0 -- fail /dev/sdb1 |
[production] |
| 11:03 |
<ema> |
depool cp3043 (cache_upload) for hardware maintenance T179953 |
[production] |
| 10:58 |
<godog> |
update compiler facts |
[production] |
| 10:53 |
<jynus> |
stop db2038 and db2047 |
[production] |
| 10:46 |
<marostegui> |
Deploy schema change on db1077 with replication, this will generate lag on labs s3 T191316 T192926 T89737 T195193 |
[production] |
| 10:42 |
<marostegui> |
Stop replication on db1077 to drop triggers on db1124:3313 - T192926 |
[production] |
| 10:39 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1077 for alter table (duration: 00m 50s) |
[production] |
| 10:28 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1123 after alter table (duration: 00m 50s) |
[production] |
| 10:01 |
<moritzm> |
rolling reboot of sca* for "lazy fpu" kernel updates |
[production] |
| 09:44 |
<_joe_> |
stopping all cronjobs via a puppet run on terbium, T192092 |
[production] |