2351-2400 of 10000 results (34ms)
2016-02-11 ยง
16:32 <thcipriani@mira> Synchronized wmf-config/CommonSettings.php: SWAT: Add $wgGraphAllowedDomains setting for future [[gerrit:269896]] (duration: 02m 13s) [production]
16:27 <elukey> Removed mc1012/mc1013 from the redis/memcached pools for maintenance. [production]
16:26 <bblack> puppet disabled on cp1065 for some live and careful experimentation... [production]
16:20 <thcipriani@mira> Synchronized wmf-config/wikitech.php: SWAT: Add wgOpenStackManagerNovaIdentityV3URI to wikitech configs [[gerrit:269558]] (duration: 02m 11s) [production]
16:16 <thcipriani@mira> Synchronized wmf-config/InitialiseSettings.php: SWAT: Add new WEF enwiki IP rate limit exception [[gerrit:269862]] (duration: 02m 13s) [production]
16:11 <thcipriani@mira> Synchronized wmf-config/InitialiseSettings.php: SWAT: wgRCWatchCategoryMembership true on wikisource [[gerrit:264735]] (duration: 02m 14s) [production]
15:43 <elukey> Add mc1010/mc1011 back to the redis/memcached pools after maintenance. [production]
15:24 <elukey> puppet re-enabled on mc1010/mc1011 [production]
14:41 <elukey> Stopped puppet, memcached, redis on mc1010/mc1011 for maintenance [production]
14:07 <elukey> removed mc1010/mc1011 from the redis/memcached pools for maintenance [production]
14:01 <urandom> `nodetool stop -- COMPACTION' on restbase1002.eqiad to free disk space (current state: https://phabricator.wikimedia.org/P2593) [production]
13:47 <jynus> backuping, reimaging, restarting and defragmenting db1024 (old s2 master) [production]
13:37 <paravoid> downgrading berkelium to Linux 3.19 & rebooting [production]
13:34 <apergos> reenabled scap runs (removed flag) [production]
13:21 <jynus> migrating dbstore1001 to the new s2-master [production]
13:00 <paravoid> upgrading backup LVS servers to Linux 4.4.0 across all sites [production]
12:42 <elukey> Add mc1008/mc1009 back into redis/memcached pools after maintenance [production]
12:40 <moritzm> uploaded nodejs 4.3.0 for jessie-wikimedia to carbon [production]
12:13 <elukey> re-enabled puppet on mc1009 [production]
12:07 <elukey> enabled puppet on mc1008 [production]
11:54 <_joe_> stopped scap runs for now (touched sync.flag) [production]
11:52 <moritzm> updated xenon, praseodymium and cerium to nodejs 4.3.0 [production]
11:45 <apergos> removed scap3 deb install from all hosts in prod [production]
11:33 <elukey> disabled puppet, memcached, redis on mc1008/mc1009 for maintenance [production]
11:12 <jynus> moving pending s2 slaves to the new master [production]
10:52 <elukey> Correction about mc1008/mc1009 - puppet not disabled, will be done later on. [production]
10:50 <elukey> removed mc1008/mc1009 from redis/memcached pools. Puppet disabled. [production]
10:03 <godog> ms-be2018 / ms-be2019 swift weight to 3500 [production]
09:52 <elukey> re-added mc1007.eqiad back into redis/memcached pools after maintenance [production]
09:39 <moritzm> depooled elastic1023 for hardware problems (T126586) [production]
09:31 <elukey> re-enabled puppet on mc1007.eqiad [production]
09:04 <jynus> restarting HHVM on all running mediawiki job queue processors [production]
08:46 <_joe_> enabled puppet on mw1017, was disabled with no reason [production]
08:45 <elukey> Disabling puppet, redis, memcached on mc1007 for maintenance [production]
08:33 <elukey> removed mc1007 from the redis/memcached pool for Jessie migration [production]
06:56 <_joe_> removing content from sd{d,j}1 written on the root partition on ms-be1008 [production]
06:39 <_joe_> stipped swift on ms-be1008, disk full because object were being written to the root partition [production]
06:16 <bd808> Re-enabled puppet on logstash1003 and forced pupept run [production]
06:02 <bd808> Raised logstash mem limit to -Xms512m on logstash1003 [production]
05:57 <bd808> disabled puppet on logstash1003 to debug configuration [production]
05:40 <bd808> logstash process on logstash1003 flapping; continuing to investigate [production]
05:25 <bd808> Restarted logstash on logstash1003; killed by OOMKiler [production]
02:16 <ori> restarted HHVM on mw1130; locked up (T89912) [production]
02:15 <bd808> Fetched 1271d36 to tin:/srv/mediawiki-staging; should have been updated by sync-wikiversions [production]
01:58 <tgr@mira> rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.13 again [production]
01:47 <mutante> labservices1001 - out of disk [production]
01:46 <mutante> ms-be1008 - delete/gzip large syslog, out of disk [production]
01:41 <tgr@mira> Synchronized wmf-config/mobile.php: fix for T49647 (duration: 02m 15s) [production]
01:05 <krenair@mira> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/269883/1 (duration: 02m 11s) [production]
00:53 <krenair@mira> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/263051/ (duration: 02m 13s) [production]