1001-1050 of 10000 results (28ms)
2016-02-29 §
12:50 <godog> gmetad on uranium restarted following https://gerrit.wikimedia.org/r/273881 it will converge asap [production]
12:16 <gehel> elastic2001.codfw.wmnet: upgrading to 1.7.5, shipping logs to logstash (T122697, T109101) [production]
11:57 <jynus> reimaging dbproxy1005 with jessie before its put into production (potential alerts due to new role) [production]
10:46 <ema> hhvm restarted on mw1119 [production]
10:25 <jynus> powercycling mw1140, almost 100% unresponsive, OOM probable cause [production]
09:58 <godog> bootstrap restbase1009-b T95253 [production]
02:32 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Feb 29 02:32:43 UTC 2016 (duration 7m 43s) [production]
02:25 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.14) (duration: 11m 12s) [production]
2016-02-28 §
22:34 <legoktm@tin> Synchronized php-1.27.0-wmf.14/includes/parser/ParserCache.php: Include ParserCache::save() backtrace in MF pollution debug log - T124356 (duration: 00m 50s) [production]
07:49 <apergos> restarted logmsbot on neon [production]
02:32 <l10nupdate@tin> ResourceLoader cache refresh completed at Sun Feb 28 02:32:00 UTC 2016 (duration 7m 42s) [production]
02:24 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.14) (duration: 10m 24s) [production]
00:59 <ori_> Restarted logmsgbot on neon; it lost its IRC connection and did not recover it. [production]
2016-02-27 §
23:19 <legoktm> 23:19:43 Synchronized php-1.27.0-wmf.14/extensions/wikihiero/: Revert "Compress PNGs with zopflipng" - https://gerrit.wikimedia.org/r/273661 (duration: 00m 46s) [production]
22:39 <legoktm> legoktm just deployed https://gerrit.wikimedia.org/r/#/c/273655/ (MF cache pollution debug log) for T124356 [production]
21:19 <urandom> issuing test repair on cerium (restbase staging), wp_parsoid html keyspace, (-w 5) : T108611 [production]
20:08 <krinkle@tin> Synchronized php-1.27.0-wmf.14/includes/mime.types: Iec9459c921: Unbreak ULS fonts via wmfstatic (duration: 00m 41s) [production]
19:32 <ori@tin> Synchronized php-1.27.0-wmf.14/includes/user/User.php: Revert I43cde3a48: Prevent duplicate memcached lookups for user record (T128246) (duration: 00m 56s) [production]
04:15 <ori> On silver: commented out exception from I3eae28a58 to work around T128246 [production]
03:53 <bd808> test [production]
03:47 <bd808> Tried to fix CAS error for Labslogbot [production]
03:36 <bd808> Backfilled missing missing https://wmflabs.org/sal/production data (2016-02-21T14:39 to 2016-02-24T16:53) [production]
03:14 <urandom> bootstrap of restbase1008-a.eqiad.wmnet complete; begining `nodetool cleanup' of 1003, 1004-a, and 1008-b : T95253 [production]
02:31 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Feb 27 02:31:10 UTC 2016 (duration 7m 44s) [production]
02:23 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.14) (duration: 10m 24s) [production]
01:50 <twentyafterfour> updated phabricator/tools to https://phabricator.wikimedia.org/rPHTO1433a20886a4168d41b81b5ecf5e29687678f2e2 (applying fix for T127524 ) [production]
00:52 <aaron@tin> Synchronized php-1.27.0-wmf.14/includes/filebackend/FileBackendMultiWrite.php: 7ca624b8b93 (duration: 00m 47s) [production]
2016-02-26 §
22:20 <mutante> powercycle mw1130 [production]
20:42 <demon@tin> Synchronized wmf-config/InitialiseSettings.php: update wmf address in echo footer thingie (duration: 00m 59s) [production]
20:29 <hashar> Restarting Jenkins [production]
20:04 <urandom> issuing test repair on cerium (restbase staging), keyspace : T108611 [production]
18:41 <ori@tin> Synchronized php-1.27.0-wmf.14/includes/user/User.php: I43cde3a48: Prevent duplicate memcached lookups for user record (duration: 01m 02s) [production]
16:46 <chasemp> labstore1001 'mdadm --manage /dev/md126 --add /dev/sdaf' [production]
15:51 <jynus> shutting down mariadb on db2030 to clone from db1009 [production]
15:24 <urandom> forcing puppet run on restbase1009.eqiad.wmnet [production]
15:24 <urandom> re-enabling puppet on restbase1009.eqiad.wmnet [production]
15:20 <jynus> performing backup of m5-master mysql data [production]
15:17 <urandom> blocking CQL native port on restbase1009.eqiad.wmnet : https://phabricator.wikimedia.org/P2677 [production]
15:14 <urandom> disabling puppet on restbase1009.eqiad to preserve local changes during a quick experiment [production]
15:03 <hashar> Switched MediaWiki core npm test to Nodepool instance T119143 [production]
13:59 <krinkle@tin> Synchronized wmf-config/InitialiseSettings.php: T99096: Enable wmgUseWmfstatic on remaining wikis (duration: 00m 50s) [production]
13:54 <moritzm> rebooting lithium for kernel update [production]
13:26 <godog> launch swiftrepl continuous replication for unsharded containers on ms-fe1003 T128096 [production]
12:31 <elukey> added mc1017/mc1018 back to the redis/memcached pools after maintenance [production]
11:42 <godog> run swiftrepl eqiad -> codfw for unsharded containers [production]
11:01 <elukey> removed mc1018/1017 from the redis memcached pools for maintenance [production]
09:46 <elukey> mc1016.eqiad re-added to the memcached/redis pools after maintenance [production]
08:12 <elukey> removed mc1016.eqiad from the redis/memcached pools for maintenance [production]
08:01 <moritzm> blacklisting aufs kernel module [production]
02:32 <l10nupdate@tin> ResourceLoader cache refresh completed at Fri Feb 26 02:32:19 UTC 2016 (duration 7m 42s) [production]