2851-2900 of 10000 results (40ms)
2016-09-13 §
17:53 <volans> reimaging mw2198 that failed early today [production]
17:36 <demon@tin> Started scap: testwiki to wmf.19 + l10n bootstrap (try 2) [production]
17:24 <demon@tin> scap failed: CalledProcessError Command '/usr/local/bin/mwscript rebuildLocalisationCache.php --wiki="testwiki" --outdir="/tmp/scap_l10n_4282891950" --threads=4 --lang en --quiet' returned non-zero exit status 1 (duration: 09m 23s) [production]
17:15 <demon@tin> Started scap: testwiki to wmf.19 + l10n bootstrap [production]
16:59 <jynus> beta dbs back in rw mode [production]
16:54 <mobrovac> restbase deploy end of d10d759 [production]
16:53 <demon@tin> Synchronized multiversion/updateWikiversions: unbreak myself (duration: 00m 48s) [production]
16:41 <mobrovac> restbase deploy start of d10d759 [production]
16:25 <mobrovac> change-prop deploying d701a69 [production]
16:24 <godog> dump hhvm backtrace on mw1162 and restart hhvm, apache gets connection refused [production]
16:17 <urandom> T144826: Removing compaction rate limit, increasing compactor threads (from 10 to 20), and beginning scrub of local_group_wikipedia_T_parsoid_html.data (restbase2004-b.codfw.wmnet) [production]
16:06 <jynus> power resetting db1082 [production]
16:02 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1082 (duration: 01m 00s) [production]
15:46 <gehel> test_* indices remvoed on relforge cluster, cluster restarted [production]
15:44 <jynus> setting deployment-db1 and deployment-db1 mysqls in read only mode [production]
15:40 <gehel> shutting down relforge cluster for indices cleanup [production]
02:46 <l10nupdate@tin> ResourceLoader cache refresh completed at Tue Sep 13 02:46:54 UTC 2016 (duration 7m 12s) [production]
02:39 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.18) (duration: 17m 41s) [production]
01:58 <aaron@tin> Synchronized wmf-config/CommonSettings.php: Lower $wgMaxUserDBWriteDuration to 3 (duration: 00m 47s) [production]
01:27 <aaron@tin> Synchronized php-1.28.0-wmf.18/extensions/GlobalUsage: 1843a856c7c648a310a1983d53139e5b79ea6585 (duration: 00m 48s) [production]
01:24 <bblack> cache_upload: reverting codfw to file storage [production]
01:15 <aaron@tin> Synchronized php-1.28.0-wmf.18/extensions/GeoData: 2dedca3789266b53cbde5bd88913c01175926974 (duration: 00m 49s) [production]
01:14 <aaron@tin> Synchronized php-1.28.0-wmf.18/includes/jobqueue/utils/PurgeJobUtils.php: (no message) (duration: 00m 52s) [production]
00:50 <bblack> cache_upload: reverting eqiad to file storage [production]
00:50 <bblack> reverting eqiad to file storage [production]
00:17 <aaron@tin> Synchronized php-1.28.0-wmf.18/extensions/GeoData: 2dedca3789266b53cbde5bd88913c01175926974 (duration: 00m 48s) [production]
00:16 <aaron@tin> Synchronized php-1.28.0-wmf.18/includes/jobqueue/utils/PurgeJobUtils.php: 0419831bbae66bda1e4dca8d690b607b493b2f5f (duration: 00m 47s) [production]
2016-09-12 §
23:40 <bblack> switch cache_upload codfw to -sdeprecated_persistent... [production]
23:40 <dereckson@tin> Synchronized wmf-config/throttle.php: Women in Science throttle rules (T145115 and T145253) (duration: 00m 47s) [production]
23:33 <urandom> T144826: Restarting Cassandra on restbase2004-b.codfw.wmnet (scrub complete, re-joining cluster) [production]
23:13 <bblack> switch cache_upload eqiad to -sdeprecated_persistent... [production]
21:54 <eileen> from 3f01d93237c2b8af4fbda5e629a4a59c77dab3c1 to f381bd1c15e30e4d47fe372e128143baee6a7c7a [production]
21:17 <arlolra> For completeness, "back" in my last log is a mistake. I scap deployed the wrong --rev, but that was ultimately the version we wanted deployed anyways, so no harm no foul. (T145460) [production]
20:40 <arlolra> Parsoid back on 7c43009c [production]
20:32 <arlolra> Parsoid deploy failed, rolling back [production]
20:28 <urandom> T143226: Perform major compaction on local_group_wikipedia_T_parsoid_html.data, restbase1015-b.eqiad.wmnet [production]
20:28 <urandom> T143226: Perform major compaction on local_group_wikipedia_T_parsoid_html.data, restbase1014-b.eqiad.wmnet [production]
20:28 <urandom> T143226: Perform major compaction on local_group_wikipedia_T_parsoid_html.data, restbase1009-b.eqiad.wmnet [production]
20:21 <mobrovac> change-prop deploying 86a60b3 [production]
20:13 <arlolra> starting Parsoid deploy [production]
19:52 <demon@tin> Synchronized multiversion/getMWVersion: for dumps <3 (duration: 00m 46s) [production]
19:44 <urandom> T143226: Perform major compaction on local_group_wikipedia_T_parsoid_html.data, restbase1013-c.eqiad.wmnet [production]
19:44 <urandom> T143226: Perform major compaction on local_group_wikipedia_T_parsoid_html.data, restbase1012-c.eqiad.wmnet [production]
19:44 <urandom> !log T143226: Perform major compaction on local_group_wikipedia_T_parsoid_html.data, restbase1008-c.eqiad.wmnet [production]
19:42 <urandom> T143226: Perform major compaction on local_group_wikipedia_T_parsoid_html.data, restbase1011-c.eqiad.wmnet [production]
19:42 <urandom> T143226: Perform major compaction on local_group_wikipedia_T_parsoid_html.data, restbase1010-c.eqiad.wmnet [production]
19:42 <urandom> T143226: Perform major compaction on local_group_wikipedia_T_parsoid_html.data, restbase1007-c.eqiad.wmnet [production]
19:13 <urandom> T133805: Restarting Cassandra to apply G1 region size of 32M on restbase1013-a.eqiad.wmnet [production]
19:12 <urandom> T133805: Disabling Puppet for GC experiment on restbase1013.eqiad.wmnet [production]
19:10 <thcipriani@tin> Synchronized static/images/project-logos: SWAT: [[gerrit:310056|Fix HD logos for hewiki (T145017)]] (duration: 00m 48s) [production]