2351-2400 of 10000 results (21ms)
2016-02-10 §
01:12 <catrope@mira> Synchronized docroot/noc/conf/highlight.php: Remove ob_start() from highlight.php (duration: 02m 13s) [production]
00:46 <RoanKattouw> Running updateCollation.php on nlwiki [production]
00:31 <catrope@mira> Synchronized wmf-config/CommonSettings.php: BetaFeatures wmg->wg rename, part 2 (duration: 02m 13s) [production]
00:28 <catrope@mira> Synchronized wmf-config/InitialiseSettings.php: Set collation to uca-nl on nlwiki; add Recherche: to content namespaces on frwikiversity; BetaFeatures wmg->wg rename (duration: 02m 12s) [production]
00:20 <catrope@mira> Synchronized wmf-config/InitialiseSettings.php: Increase completion suggester replicas for busy wikis (duration: 02m 11s) [production]
00:18 <catrope@mira> Synchronized wmf-config/logging.php: Reduce Kafka timeouts (duration: 02m 13s) [production]
00:11 <catrope@mira> Synchronized wmf-config/InitialiseSettings.php: Test HTML stripping in production mobile beta (duration: 02m 12s) [production]
2016-02-09 §
23:55 <jynus> restarting jobrunner and jobchron [production]
23:38 <jynus> setting db1018's binlog_format as STATEMENT [production]
23:30 <jynus@mira> Synchronized wmf-config/db-eqiad.php: Disable read only mode for s2 after its master failover (duration: 02m 09s) [production]
23:27 <_joe_> disabled puppet on mc1004, added "bind 0.0.0.0" to its redis config, restarted redis (T126395) [production]
23:23 <jynus> setting db1018 in read/write mode [production]
23:22 <jynus@mira> Synchronized wmf-config/db-eqiad.php: Actual mediawiki master failover (duration: 02m 14s) [production]
23:19 <bd808> Changed /src/mediawiki/wikiverisons.php on mw1017 (X-Wikimedia-Debug) to set all wikis to 1.27.0-wmf.13 [production]
23:18 <jynus> setting db1024 in read only mode [production]
23:16 <jynus@mira> Synchronized wmf-config/db-eqiad.php: Enabling read only mode for s2 before its master failover (duration: 02m 14s) [production]
23:09 <jynus> setting up circular replication between db1018 and db1024 for potential rollback [production]
23:02 <jynus> changing topology of s2 slaves in preparation for master failover [production]
22:04 <bd808@mira> Synchronized wmf-config/logging.php: Monolog: reorder Monolog processors (b356eeb) (duration: 02m 15s) [production]
21:33 <Krenair> ran package upgrades on wikitech-static [production]
20:37 <bblack> restarting nginx for libssl update on cp1049.eqiad.wmnet,cp4008.ulsfo.wmnet,cp3042.esams.wmnet,cp3049.esams.wmnet [production]
20:32 <demon@mira> Finished scap: all group0 to wmf.13 (duration: 29m 45s) [production]
20:25 <bblack> cache kernel reboots done (all on '3.19.0-2-amd64 #1 SMP Debian 3.19.3-9 (2016-01-04)', except 4x canaries on '4.4.0-1-amd64 #1 SMP Debian 4.4-1~wmf1 (2016-01-26)') [production]
20:11 <bblack> cp1067, cp1071 (text, upload in eqiad) -> 4.4 canaries (rebooting over the next ~8 mins or so) [production]
20:02 <demon@mira> Started scap: all group0 to wmf.13 [production]
19:55 <hoo> Updated operations/dumps/dcat on snapshot1003 from 0a71deb232 to 92ab37d94e [production]
19:37 <demon@mira> Finished scap: pruning tons of stale branches + sync wmf.13 files for later + testwiki to wmf.13 to build l10n cache (try 2) (duration: 27m 24s) [production]
19:35 <bblack> cp3048 (upload esams) rebooting -> kernel 4.4 canary [production]
19:13 <mutante> gerrit - add ppchelko to mediawiki-services [production]
19:11 <bblack> cp4006 (upload ulsfo) rebooting -> kernel 4.4 canary [production]
19:09 <demon@mira> Started scap: pruning tons of stale branches + sync wmf.13 files for later + testwiki to wmf.13 to build l10n cache (try 2) [production]
19:07 <demon@mira> scap failed: CalledProcessError Command '/usr/local/bin/mwscript rebuildLocalisationCache.php --wiki="testwiki" --outdir="/tmp/scap_l10n_2315818744" --threads=10 --lang en --quiet' returned non-zero exit status 255 (duration: 00m 34s) [production]
19:07 <demon@mira> Started scap: pruning tons of stale branches + sync wmf.13 files for later + testwiki to wmf.13 to build l10n cache [production]
18:57 <yurik> deployed graphoid [production]
18:05 <jynus> bringing down db1048's mysql for cloning to db2012 [production]
17:54 <Krenair> ssh: connect to host mw1037.eqiad.wmnet port 22: Connection timed out [production]
17:53 <krenair@mira> Synchronized php-1.27.0-wmf.12/extensions/OpenStackManager: https://gerrit.wikimedia.org/r/#/c/269439/ (duration: 03m 15s) [production]
17:26 <elukey> mc1004.eqiad put back into redis/memcached pool [production]
17:23 <godog> nodetool-a removenode ec0c5a3d-2648-4933-8434-a8d163b92188 in preparation for restbase1007 bootstrap [production]
17:22 <bblack> rebooting cp1008/pinkunicorn for 4.4-rt kernel test [production]
17:19 <_joe_> powered down mw1037 [production]
17:07 <godog> start cassandra-a on restbase1007 with replace_address=10.64.0.230 [production]
16:57 <thcipriani@mira> Finished scap: SWAT: Clarify and expand messages mentioning loss of session data [[gerrit:269424]] (duration: 27m 36s) [production]
16:53 <bblack> rebooting cp1008/pinkunicorn for 4.4 kernel [production]
16:34 <jynus> reimage db2012 [production]
16:30 <thcipriani@mira> Started scap: SWAT: Clarify and expand messages mentioning loss of session data [[gerrit:269424]] [production]
16:18 <thcipriani@mira> Synchronized wmf-config: SWAT: Enable ArticlePlaceholder on test wikis [[gerrit:269399]] (duration: 01m 19s) [production]
16:15 <thcipriani> mw1037.eqiad.wmnet error during SWAT rsync: failed to set times on "/srv/mediawiki/.": Read-only file system (30) [production]
16:09 <thcipriani@mira> Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable math data type on Wikidata and everywhere [[gerrit:269398]] (duration: 02m 31s) [production]
15:59 <elukey> puppet re-enabled on kafka1012 [production]