1501-1550 of 10000 results (34ms)
2016-02-10 ยง
20:21 <demon@mira> Synchronized php-1.27.0-wmf.13/includes/interwiki/Interwiki.php: fix cache stuff (duration: 02m 18s) [production]
20:17 <apergos> cleaned up integrations slave trusty 1001,10012,10013, 1016, missed in first round. [production]
20:10 <demon@mira> rebuilt wikiversions.php and synchronized wikiversions files: rebuild [production]
20:10 <demon@mira> Synchronized multiversion/MWWikiversions.php: rm newline addition (duration: 02m 19s) [production]
20:03 <demon@mira> rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.13 [production]
19:45 <apergos> did cleanup across all integration slaves, some were very close to out of room. results: https://phabricator.wikimedia.org/P2587 [production]
19:26 <mutante> ms-be1008 - powercycling - the known XFS issue [production]
19:04 <mobrovac> restbase disabled puppet in staging, testing brotli compression which requires JAVA_OPTS tuning [production]
18:30 <mutante> puppetstoredconfigclean.rb iodine.wikimedia.org, revoke puppet cert, delete salt key on new master [production]
18:15 <mutante> iodine - shutdown, decom [production]
18:10 <mutante> iodine - schedule downtime, stop puppet, stop salt, .. [production]
17:21 <demon@mira> Synchronized multiversion/MWWikiversions.php: newlines in wikiversion.json (duration: 02m 21s) [production]
17:15 <demon@mira> Synchronized errorpages/404.html: minor html fix (duration: 02m 17s) [production]
17:12 <thcipriani@mira> Finished scap: VE, Cite, and Citoid bumps [[gerrit:269592]] [[gerrit:269593]] [[gerrit:269575]] [[gerrit:269590]] (duration: 29m 53s) [production]
17:02 <elukey> readded mc1006 back into redis/memcached pool [production]
16:42 <thcipriani@mira> Started scap: VE, Cite, and Citoid bumps [[gerrit:269592]] [[gerrit:269593]] [[gerrit:269575]] [[gerrit:269590]] [production]
16:35 <thcipriani@mira> Synchronized wmf-config/InitialiseSettings.php: SWAT: Revert "Enable Math extension on Wikitech" (duration: 02m 14s) [production]
16:24 <thcipriani@mira> Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable Math extension on Wikitech [[gerrit:269694]] (duration: 02m 14s) [production]
16:18 <thcipriani@mira> Synchronized wmf-config/InitialiseSettings.php: SWAT: Namespaces configuration on ru.wikisource [[gerrit:269702]] (duration: 02m 10s) [production]
16:09 <thcipriani@mira> Synchronized wmf-config/InitialiseSettings.php: SWAT: CX: Enable specialcx campaign [[gerrit:269416]] (duration: 02m 22s) [production]
15:51 <jynus> recreating labsdb heartbeat views to correctly measure lag from the new s2-master [production]
15:46 <elukey> re-enabled puppet on mc1006.eqiad [production]
15:42 <ostriches> gerrit: flushed all caches, things will be slow for a bit while they warm [production]
15:22 <elukey> disabled puppet/memcached/redis on mc1006.eqiad [production]
15:02 <bblack> SPDY disable for cache_text: test starts in a few minutes! - https://phabricator.wikimedia.org/T125979 [production]
14:48 <elukey> removed mc1006 from the redis/memcached pool for Jessie migration [production]
13:48 <godog> restbase1002 nodetool setstreamthroughput 500 [production]
13:24 <elukey> adding mc1005.eqiad back into service (redis/memcached) [production]
12:51 <jynus> restarting hhvm at mw1015 - db errors continue [production]
12:42 <Krinkle> Purged MediaWiki/wmfstatic/* metrics in Graphite (spurious test data) [production]
12:35 <krinkle@mira> Synchronized w/static.php: (no message) (duration: 02m 11s) [production]
12:33 <moritzm> disabling puppet on mw1001-1009, mw1011-1016 to enable ferm in batches [production]
11:51 <jynus> restarting mw1015 jobqueue/chron processing again [production]
11:44 <godog> restbase1001 nodetool setstreamthroughput 500 [production]
11:37 <elukey> re-enabled puppet on mc1005.eqiad [production]
11:36 <godog> restbase1001 nodetool setstreamthroughput 400 [production]
11:16 <godog> restbase1007 nodetool-a setstreamthroughput 350 [production]
11:02 <jynus> enabling semi-sync replication fo s2 slaves [production]
10:58 <elukey> disabling puppet, redis and memcached on mc1005 (preparation for Jessie migration) [production]
10:44 <_joe_> removing mw1051-mw1070 from the appservers pool (T126242) [production]
10:26 <elukey> Removing mc1005.eqiad from the redis/memcached pools [production]
10:20 <godog> ms-be2016 / ms-be2017 swift weight to 3500 [production]
08:24 <godog> removenode restbase1007-a finished, start cassandra-a on restbase1007 for bootstrap [production]
07:54 <apergos> cleared out a bunch of clones from /mnt/jenkins-workspace/workspace on integration-slave-trusty-1016, /mnt was full preventing jenkins from completing e.g. https://integration.wikimedia.org/ci/job/operations-puppet-typos/50458/console [production]
07:20 <ottomata> puppet disabled on analytics1027 til tomorrow while cron is disabled and CirrusSearchRequestSet backfills into Hadoop from kafka [production]
06:19 <mutante> integration-slave-trusty-1014 - out of disk ? jenkins voted things -1 because it had no space left on device [production]
02:01 <jynus@mira> Synchronized wmf-config/db-eqiad.php: Reducing new s2 master weight for reads (duration: 02m 15s) [production]
01:49 <jynus@mira> Synchronized wmf-config/db-codfw.php: Updating new master on codfw configuration (duration: 02m 15s) [production]
01:41 <jynus> starting pt-heartbeat on db1018 [production]
01:23 <jynus> started db1048 replication. For some reason, replication was stopped. Need further investigation. [production]