851-900 of 10000 results (28ms)
2015-08-10 §
14:42 <ottomata> restarted all varnishkafka instances to pick up proper confs (puppet should have done this!) [production]
14:29 <ottomata> starting upgrade of existing kafka cluster to 0.8.2.1 jessie - https://etherpad.wikimedia.org/p/kafka_0.8.2.1_migration2 [production]
12:38 <bblack> deployed nginx-1.9.3-1+wmf2 to cp1065, cp1070, cp1071 (1x each text, upload, misc) for validation [production]
11:16 <hoo> Synchronized wmf-config/: Revert "Set dispatchBatchChunkFactor to 10 for now" (duration: 00m 12s) [production]
11:16 <hoo> Synchronized php-1.26wmf17/extensions/Wikidata/: Revert "Set dispatchBatchChunkFactor to 10 for now" (duration: 00m 20s) [production]
09:35 <paravoid> manually firewalled backup4001 TCP on neon to temporarily stop the nsca alert storm [production]
09:12 <jynus> Synchronized wmf-config/db-eqiad.php: depool db1042 for maintenance (duration: 00m 12s) [production]
08:43 <_joe_> manually running logrotate on iridium [production]
07:44 <godog> reboot ms-be2006, xfs hosed [production]
07:27 <akosiaris> rebooting backup4001 [production]
07:20 <jynus> schema change on testwikidatawiki [production]
06:52 <jynus> Synchronized wmf-config/db-eqiad.php: Increase db1035 weight (duration: 00m 13s) [production]
05:31 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Aug 10 05:31:29 UTC 2015 (duration 31m 28s) [production]
03:13 <bblack> restarted apache2 on iridium JIC [production]
03:13 <bblack> rm /var/log/apache2/phabricator_access.log.1 on iridium (disk full, fixed for now) [production]
02:23 <l10nupdate@tin> LocalisationUpdate completed (1.26wmf17) at 2015-08-10 02:23:47+00:00 [production]
02:20 <l10nupdate> Synchronized php-1.26wmf17/cache/l10n: l10nupdate for 1.26wmf17 (duration: 06m 43s) [production]
2015-08-09 §
17:57 <urandom> issuing nodetool cleanup on restbase1006 [production]
12:56 <hoo> Synchronized php-1.26wmf17/extensions/Wikidata/: Set dispatchBatchChunkFactor to 10 for now (duration: 00m 20s) [production]
06:20 <twentyafterfour> restarted phabricator phd (just in case - the full partition may have caused the daemons to be in a broken state) [production]
06:17 <twentyafterfour> moved some log files on iridium into /srv/logs to free space on / [production]
05:12 <l10nupdate@tin> ResourceLoader cache refresh completed at Sun Aug 9 05:12:23 UTC 2015 (duration 12m 22s) [production]
02:23 <l10nupdate@tin> LocalisationUpdate completed (1.26wmf17) at 2015-08-09 02:23:22+00:00 [production]
02:20 <l10nupdate> Synchronized php-1.26wmf17/cache/l10n: l10nupdate for 1.26wmf17 (duration: 06m 32s) [production]
2015-08-08 §
14:58 <urandom> issuing nodetool cleanup on restbase1005 [production]
14:57 <urandom> issuing nodetool cleanup on restbase1007 [production]
05:30 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Aug 8 05:30:22 UTC 2015 (duration 30m 21s) [production]
04:20 <urandom> issuing nodetool cleanup on restbase1008 [production]
03:00 <hoo> Synchronized php-1.26wmf17/extensions/Wikidata/: Hack: Don't write change rows where LENGTH(change_info) > 65500 (duration: 00m 21s) [production]
02:26 <l10nupdate@tin> LocalisationUpdate completed (1.26wmf17) at 2015-08-08 02:26:20+00:00 [production]
02:23 <l10nupdate> Synchronized php-1.26wmf17/cache/l10n: l10nupdate for 1.26wmf17 (duration: 06m 20s) [production]
01:53 <hoo> Deleted change 237365841 as well [production]
01:37 <hoo> Deleted changes 237357747 and 237363245 from wikidata's wb_changes [production]
01:32 <ori> Synchronized php-1.26wmf17/resources/src/mediawiki.legacy/wikibits.js: I664ba9b0af: Override document.writeln to prevent it from blanking pages (duration: 00m 13s) [production]
2015-08-07 §
21:00 <krenair> Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/230227/ - should be a noop for prod (duration: 00m 12s) [production]
19:40 <hoo> Synchronized wmf-config/: Bump wgCacheEpoch for Wikidata (duration: 00m 13s) [production]
19:29 <hoo> Synchronized php-1.26wmf17/extensions/Wikidata/: Update Wikibase: Fix WB spinner, UnresolvedRedirectException handling on client (duration: 00m 21s) [production]
18:08 <gwicke> switched restbase1001 to CMS temporarily, to gather metrics; will switch back to G1GC tonight [production]
17:09 <yurik> synced kartotherian to maps-test* servers again, restarted the service [production]
16:00 <jynus> repool db1035 database with low traffic [production]
15:54 <thcipriani> Synchronized php-1.26wmf17/extensions/ContentTranslation/modules/tools/ext.cx.tools.images.js: FIX: Use .attr() to set the resource attribute of image, while adapting [[gerrit:230101]] (duration: 00m 11s) [production]
15:10 <yurik> synced kartotherian to maps-test* servers [production]
15:10 <gwicke> switched cassandra staging cluster back to G1GC / default puppet [production]
13:40 <moritzm> install pcre security updates on elastic*, analytics*, wtp*, db* and es* [production]
13:39 <urandom> starting nodetool cleanup on restbase1004 (see: T108083) [production]
13:37 <urandom> starting nodetool cleanup on restbase1002 (see: T108083) [production]
12:07 <moritzm> restarted HHVM on jobrunners/imagescalers in eqiad/codfw for libtidy/PCRE security updates [production]
09:24 <akosiaris> Synchronized wmf-config/PoolCounterSettings-eqiad.php: (no message) (duration: 00m 12s) [production]
09:02 <akosiaris> disabled helium as a poolcounter temporarily while applying base::firewall again [production]
09:01 <akosiaris> Synchronized wmf-config/PoolCounterSettings-eqiad.php: (no message) (duration: 00m 12s) [production]