4901-4950 of 10000 results (37ms)
2015-06-29 §
12:06 <YuviPanda> excluded maps, mwoffliner and video project from rsync of broken FS to speed it up [production]
11:59 <YuviPanda> interupt rsync on labstore1001 to prevent it from copying mwofflienr files [production]
11:00 <_joe_> shutting down etcd1003, cleaning exported resources [production]
10:32 <_joe_> effectively removing etcd1003 from the cluster [production]
10:17 <_joe_> starting removal of etcd1003 from the etcd cluster [production]
08:49 <_joe_> joined conf1003 to the etcd cluster [production]
08:21 <jynus> Synchronized wmf-config/db-eqiad.php: Depool db1022 for reinstall (duration: 00m 12s) [production]
08:12 <_joe_> adding conf1002 to the etcd cluster as a member [production]
07:46 <akosiaris> disabling ntp everywhere expect selected hosts in anticipation for the leap second [production]
04:51 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Jun 29 04:51:48 UTC 2015 (duration 51m 47s) [production]
03:08 <jgage> jmxtrans filled disks on all kafka brokers, 21GB log files. removed logs and restarted services. [production]
02:23 <LocalisationUpdate> completed (1.26wmf11) at 2015-06-29 02:23:47+00:00 [production]
02:20 <l10nupdate> Synchronized php-1.26wmf11/cache/l10n: (no message) (duration: 05m 53s) [production]
00:52 <springle> restart eventlogging auto-purge on m4 [production]
00:51 <springle> restart replication on dbstore2002 [production]
00:00 <springle> pausing replication on dbstore2002 [production]
2015-06-28 §
23:51 <ori> Synchronized php-1.26wmf11/extensions/CentralNotice/modules/ext.centralNotice.bannerController/bannerController.js: I6ffdc977e87: Parse older format of Geo cookies (duration: 00m 13s) [production]
04:30 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Jun 28 04:30:54 UTC 2015 (duration 30m 53s) [production]
02:20 <LocalisationUpdate> completed (1.26wmf11) at 2015-06-28 02:20:52+00:00 [production]
02:17 <l10nupdate> Synchronized php-1.26wmf11/cache/l10n: (no message) (duration: 05m 56s) [production]
2015-06-27 §
23:30 <bd808> Deleted corrupt shards on logstash1004 and logstash1005. Recovery in process [production]
20:12 <ori> Delegated full access to Google Webmaster Tools for myself (olivneh@). [production]
04:58 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Jun 27 04:58:46 UTC 2015 (duration 58m 45s) [production]
02:23 <LocalisationUpdate> completed (1.26wmf11) at 2015-06-27 02:23:40+00:00 [production]
02:20 <l10nupdate> Synchronized php-1.26wmf11/cache/l10n: (no message) (duration: 05m 46s) [production]
2015-06-26 §
23:57 <bd808> Logstash log ingestion working again after forcing recovery of replicas for logstash-2015.06.26; new logs were being rejected with only a primary shard available [production]
23:54 <bd808> re-enabled allocation on logstash elasticsearch cluster [production]
23:05 <bblack> restarted gitblit on antimony, AGAIN [production]
22:57 <mutante> restarted gitblit [production]
22:44 <catrope> Synchronized php-1.26wmf11/extensions/Flow: Temporarily make subpages in Flow-occupied namespaces non-Flow again (duration: 00m 14s) [production]
22:36 <bd808> set indices.recovery.concurrent_streams to 4 on logstash ES cluster [production]
22:36 <godog> set indices.recovery.max_bytes_per_sec to 10mb on logstash ES cluster [production]
22:25 <godog> set indices.recovery.max_bytes_per_sec to 50mb on logstash ES cluster [production]
22:25 <jamesofur> Reset email address of User:Chwms identity verified in person at editathon [production]
22:09 <bd808> restarted logstash on logstash1001 [production]
21:10 <urandom> taking xenon down to be rebootstrapped [production]
20:10 <bd808> Deleted 4 corrupt indices (logstash-2015.05.30 logstash-2015.05.31 logstash-2015.06.03 logstash-2015.06.06) on logstash1004 [production]
19:58 <bd808> stopping elasticsearch on logstash1004 to cleanup corrupt shards [production]
17:05 <mutante> zirconium - manual cleanup, removing planet [production]
17:04 <godog> reverted cronolog puppetmaster patch, restarting apache [production]
14:17 <Krenair> Deployed patch for T103391 [production]
12:23 <krenair> Synchronized wmf-config/wikitech.php: https://gerrit.wikimedia.org/r/#/c/221105/ (duration: 00m 12s) [production]
12:18 <_joe_> added conf1001 to the etcd cluster [production]
07:57 <krinkle> Synchronized php-1.26wmf11/extensions/Popups: T103610 (duration: 00m 11s) [production]
06:04 <LocalisationUpdate> ResourceLoader cache refresh completed at Fri Jun 26 06:04:14 UTC 2015 (duration 4m 13s) [production]
05:22 <twentyafterfour> restarted apache on iridium to fix phabricator fatal [production]
02:33 <LocalisationUpdate> completed (1.26wmf11) at 2015-06-26 02:33:33+00:00 [production]
02:30 <l10nupdate> Synchronized php-1.26wmf11/cache/l10n: (no message) (duration: 05m 36s) [production]
00:51 <gwicke> reverted restbase1001 canary to 90817c2a [production]
00:36 <ori> Synchronized php-1.26wmf11/extensions/SyntaxHighlight_GeSHi: I0e5f2d3b2: Updated mediawiki/core Project: mediawiki/extensions/SyntaxHighlight_GeSHi (duration: 00m 11s) [production]