|
2018-07-10
ยง
|
| 14:31 |
<marostegui> |
Set disk #0 offline for replacement - T199056 |
[production] |
| 14:17 |
<zfilipin@deploy1001> |
Started scap: testwiki to php-1.32.0-wmf.12 and rebuild l10n cache |
[production] |
| 14:15 |
<moritzm> |
installing tiff security updates on jessie |
[production] |
| 13:53 |
<moritzm> |
installing ruby-sprockets security updates |
[production] |
| 13:48 |
<moritzm> |
installing subversion updates from jessie 9.11 point release |
[production] |
| 13:40 |
<moritzm> |
rolling restart of thumbor to pick up new exiv2/openssl |
[production] |
| 13:29 |
<thcipriani@deploy1001> |
Pruned MediaWiki: 1.32.0-wmf.5 (duration: 03m 13s) |
[production] |
| 13:26 |
<moritzm> |
installing cups security updates on jessie |
[production] |
| 13:25 |
<thcipriani@deploy1001> |
Pruned MediaWiki: 1.32.0-wmf.4 (duration: 06m 18s) |
[production] |
| 13:14 |
<thcipriani@deploy1001> |
Synchronized scap/plugins/clean.py: no op sync for consistancy [[gerrit:441920|Scap clean: remove remote cache directory]] (duration: 00m 51s) |
[production] |
| 12:53 |
<aaron@deploy1001> |
Synchronized php-1.32.0-wmf.10/includes/libs/objectcache/ReplicatedBagOStuff.php: 4ad6b70ba132c66e14a706eae240887885946a42 (duration: 00m 51s) |
[production] |
| 12:24 |
<aaron@deploy1001> |
Synchronized php-1.32.0-wmf.10/extensions/SpamBlacklist/includes/SpamBlacklist.php: 08a2153f7aa (duration: 00m 51s) |
[production] |
| 12:20 |
<aaron@deploy1001> |
Synchronized php-1.32.0-wmf.12/extensions/SpamBlacklist/includes/SpamBlacklist.php: 583dc7a92f9b (duration: 00m 51s) |
[production] |
| 12:06 |
<aaron@deploy1001> |
Synchronized wmf-config/mc.php: Revert "Make all non-test wikis write to both nutcracker and mcrouter" (duration: 00m 56s) |
[production] |
| 11:56 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@2674971]: Roll out cassandra-driver@3.5.0 to restbase2001 (duration: 02m 52s) |
[production] |
| 11:53 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@2674971]: Roll out cassandra-driver@3.5.0 to restbase2001 |
[production] |
| 11:43 |
<arturo> |
updated compiler facts `PUPPET_MASTERS=puppetmaster1001.eqiad.wmnet PUPPET_COMPILER=compiler02.puppet3-diffs.eqiad.wmflabs modules/puppet_compiler/files/compiler-update-facts` |
[production] |
| 11:32 |
<aaron@deploy1001> |
Synchronized wmf-config/mc.php: Make all non-test wikis write to both nutcracker and mcrouter (duration: 00m 51s) |
[production] |
| 11:25 |
<dcausse@deploy1001> |
Synchronized ./wmf-config/InitialiseSettings.php: [cirrus] cleanup unused config vars 2/2 (duration: 01m 40s) |
[production] |
| 11:24 |
<moritzm> |
installing PHP 7 security updates on stretch |
[production] |
| 11:09 |
<dcausse@deploy1001> |
Synchronized ./wmf-config/: [cirrus] cleanup unused config vars 1/2 (duration: 00m 53s) |
[production] |
| 10:27 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@d724ad1]: Fix up the invalid Vary header, take 2, checer timed out (duration: 06m 39s) |
[production] |
| 10:20 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@d724ad1]: Fix up the invalid Vary header, take 2, checer timed out |
[production] |
| 10:20 |
<ppchelko@deploy1001> |
deploy aborted: Fix up the invalid Vary header (duration: 12m 50s) |
[production] |
| 10:07 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@d724ad1]: Fix up the invalid Vary header |
[production] |
| 10:07 |
<moritzm> |
restarting thumbor on thumbor1001 to pick up exiv2 security updates |
[production] |
| 10:03 |
<elukey> |
restart analytics100[1,2]'s hadoop resource managers, some I/O socket errors after the ip6 interface change |
[production] |
| 09:34 |
<elukey> |
forced umount of /mnt/hdfs on stat1004, several processes hang for it (causing load) and transport not connected |
[production] |
| 09:25 |
<godog> |
swift eqiad-prod add ms-be1036 back gradually - T196873 |
[production] |
| 09:05 |
<moritzm> |
installing libsoup security updates on jessie/stretch |
[production] |
| 08:38 |
<moritzm> |
installing libjpeg-turbo security updates on trusty |
[production] |
| 08:37 |
<godog> |
drain and restart cassandra-a on restbase2001 to test a restart |
[production] |
| 08:34 |
<elukey> |
rolling restart of AQS to apply the new config |
[production] |
| 08:09 |
<godog> |
disable puppet on hosts running cassandra before merging 444247 and 443114 |
[production] |
| 08:08 |
<moritzm> |
installing openslp security updates on trusty |
[production] |
| 07:58 |
<moritzm> |
installing ntp security updates on trusty (may trigger some Icinga warnings about clocks, these recover after a while) |
[production] |
| 07:33 |
<twentyafterfour> |
deploying fix for phabricator userpage having hard-coded my username |
[production] |
| 06:53 |
<twentyafterfour> |
deployed rPHEX03173dd0097451f60faa3a1705abee58a9fe4c5f |
[production] |
| 06:32 |
<marostegui> |
Optimize wbc_entity_usage on arwiki cawiki huwiki rowiki ukwiki on s7 codfw master (db2040) with replication, this will generate lag on s7 codfw - T187521 |
[production] |
| 06:23 |
<marostegui> |
Deploy schema change on codfw s7 master (db2040) with replication, this will generate lag on s7 codfw T146591 T197891 T196379 |
[production] |
| 06:17 |
<marostegui> |
Deploy schema change on s8 primary master (db1071) T146591 T197891 T196379 |
[production] |
| 06:06 |
<marostegui> |
Deploy schema change on dbstore1002:s8 T146591 T197891 T196379 |
[production] |
| 06:02 |
<marostegui> |
Deploy schema change on codfw s8 master (db2045) with replication, this will generate lag on s8 codfw T146591 T197891 T196379 |
[production] |
| 05:40 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1091 after alter table (duration: 00m 50s) |
[production] |
| 05:39 |
<marostegui> |
Deploy schema change on s4 primary master (db1068) T146591 T197891 T196379 |
[production] |
| 05:36 |
<marostegui> |
Deploy schema change on db1091 T146591 T197891 T196379 |
[production] |
| 05:36 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1091 for alter table (duration: 00m 50s) |
[production] |
| 05:32 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1081 after alter table (duration: 00m 49s) |
[production] |
| 05:27 |
<marostegui> |
Deploy schema change on db1081 T146591 T197891 T196379 |
[production] |
| 05:27 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1081 for alter table (duration: 00m 50s) |
[production] |