2016-02-29
§
|
12:50 |
<godog> |
gmetad on uranium restarted following https://gerrit.wikimedia.org/r/273881 it will converge asap |
[production] |
12:16 |
<gehel> |
elastic2001.codfw.wmnet: upgrading to 1.7.5, shipping logs to logstash (T122697, T109101) |
[production] |
11:57 |
<jynus> |
reimaging dbproxy1005 with jessie before its put into production (potential alerts due to new role) |
[production] |
10:46 |
<ema> |
hhvm restarted on mw1119 |
[production] |
10:25 |
<jynus> |
powercycling mw1140, almost 100% unresponsive, OOM probable cause |
[production] |
09:58 |
<godog> |
bootstrap restbase1009-b T95253 |
[production] |
02:32 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Feb 29 02:32:43 UTC 2016 (duration 7m 43s) |
[production] |
02:25 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.14) (duration: 11m 12s) |
[production] |
2016-02-27
§
|
23:19 |
<legoktm> |
23:19:43 Synchronized php-1.27.0-wmf.14/extensions/wikihiero/: Revert "Compress PNGs with zopflipng" - https://gerrit.wikimedia.org/r/273661 (duration: 00m 46s) |
[production] |
22:39 |
<legoktm> |
legoktm just deployed https://gerrit.wikimedia.org/r/#/c/273655/ (MF cache pollution debug log) for T124356 |
[production] |
21:19 |
<urandom> |
issuing test repair on cerium (restbase staging), wp_parsoid html keyspace, (-w 5) : T108611 |
[production] |
20:08 |
<krinkle@tin> |
Synchronized php-1.27.0-wmf.14/includes/mime.types: Iec9459c921: Unbreak ULS fonts via wmfstatic (duration: 00m 41s) |
[production] |
19:32 |
<ori@tin> |
Synchronized php-1.27.0-wmf.14/includes/user/User.php: Revert I43cde3a48: Prevent duplicate memcached lookups for user record (T128246) (duration: 00m 56s) |
[production] |
04:15 |
<ori> |
On silver: commented out exception from I3eae28a58 to work around T128246 |
[production] |
03:53 |
<bd808> |
test |
[production] |
03:47 |
<bd808> |
Tried to fix CAS error for Labslogbot |
[production] |
03:36 |
<bd808> |
Backfilled missing missing https://wmflabs.org/sal/production data (2016-02-21T14:39 to 2016-02-24T16:53) |
[production] |
03:14 |
<urandom> |
bootstrap of restbase1008-a.eqiad.wmnet complete; begining `nodetool cleanup' of 1003, 1004-a, and 1008-b : T95253 |
[production] |
02:31 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sat Feb 27 02:31:10 UTC 2016 (duration 7m 44s) |
[production] |
02:23 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.14) (duration: 10m 24s) |
[production] |
01:50 |
<twentyafterfour> |
updated phabricator/tools to https://phabricator.wikimedia.org/rPHTO1433a20886a4168d41b81b5ecf5e29687678f2e2 (applying fix for T127524 ) |
[production] |
00:52 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.14/includes/filebackend/FileBackendMultiWrite.php: 7ca624b8b93 (duration: 00m 47s) |
[production] |
2016-02-26
§
|
22:20 |
<mutante> |
powercycle mw1130 |
[production] |
20:42 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: update wmf address in echo footer thingie (duration: 00m 59s) |
[production] |
20:29 |
<hashar> |
Restarting Jenkins |
[production] |
20:04 |
<urandom> |
issuing test repair on cerium (restbase staging), keyspace : T108611 |
[production] |
18:41 |
<ori@tin> |
Synchronized php-1.27.0-wmf.14/includes/user/User.php: I43cde3a48: Prevent duplicate memcached lookups for user record (duration: 01m 02s) |
[production] |
16:46 |
<chasemp> |
labstore1001 'mdadm --manage /dev/md126 --add /dev/sdaf' |
[production] |
15:51 |
<jynus> |
shutting down mariadb on db2030 to clone from db1009 |
[production] |
15:24 |
<urandom> |
forcing puppet run on restbase1009.eqiad.wmnet |
[production] |
15:24 |
<urandom> |
re-enabling puppet on restbase1009.eqiad.wmnet |
[production] |
15:20 |
<jynus> |
performing backup of m5-master mysql data |
[production] |
15:17 |
<urandom> |
blocking CQL native port on restbase1009.eqiad.wmnet : https://phabricator.wikimedia.org/P2677 |
[production] |
15:14 |
<urandom> |
disabling puppet on restbase1009.eqiad to preserve local changes during a quick experiment |
[production] |
15:03 |
<hashar> |
Switched MediaWiki core npm test to Nodepool instance T119143 |
[production] |
13:59 |
<krinkle@tin> |
Synchronized wmf-config/InitialiseSettings.php: T99096: Enable wmgUseWmfstatic on remaining wikis (duration: 00m 50s) |
[production] |
13:54 |
<moritzm> |
rebooting lithium for kernel update |
[production] |
13:26 |
<godog> |
launch swiftrepl continuous replication for unsharded containers on ms-fe1003 T128096 |
[production] |
12:31 |
<elukey> |
added mc1017/mc1018 back to the redis/memcached pools after maintenance |
[production] |
11:42 |
<godog> |
run swiftrepl eqiad -> codfw for unsharded containers |
[production] |
11:01 |
<elukey> |
removed mc1018/1017 from the redis memcached pools for maintenance |
[production] |
09:46 |
<elukey> |
mc1016.eqiad re-added to the memcached/redis pools after maintenance |
[production] |
08:12 |
<elukey> |
removed mc1016.eqiad from the redis/memcached pools for maintenance |
[production] |
08:01 |
<moritzm> |
blacklisting aufs kernel module |
[production] |
02:32 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri Feb 26 02:32:19 UTC 2016 (duration 7m 42s) |
[production] |