651-700 of 10000 results (31ms)
2016-03-14 §
10:27 <mobrovac> restbase restarting restbase in prod to apply https://gerrit.wikimedia.org/r/276728 [production]
10:13 <mobrovac> restbase deployed 3bedb8f on restbase101[01].eqiad.wmnet [production]
10:06 <mobrovac> restbase stopping restbase on rb100[12] before full deprovisioning [production]
09:46 <godog> depool restbase1001 / restbase1002 before deprovisioning [production]
05:19 <ori> Restarting HHVM on mw1025 with known-bad config option 'hhvm.enable_reusable_tc = true' for debugging [production]
02:38 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Mar 14 02:38:54 UTC 2016 (duration 8m 41s) [production]
02:30 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.16) (duration: 13m 23s) [production]
2016-03-13 §
02:39 <l10nupdate@tin> ResourceLoader cache refresh completed at Sun Mar 13 02:39:06 UTC 2016 (duration 8m 35s) [production]
02:30 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.16) (duration: 13m 24s) [production]
2016-03-12 §
17:11 <hoo> Updated operations/dumps/dcat on snapshot1003 from 92ab37d94e to e97408df39 [production]
04:44 <legoktm@tin> Synchronized php-1.27.0-wmf.16/extensions/CharInsert/: Revert "Remove inline event handler js from charinsert" - https://gerrit.wikimedia.org/r/#/c/276932/ T129524 (duration: 00m 29s) [production]
02:38 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Mar 12 02:38:16 UTC 2016 (duration 8m 45s) [production]
02:29 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.16) (duration: 12m 52s) [production]
00:55 <aaron@tin> Synchronized rpc/RunJobs.php: 7a8bd37247b7dfb (duration: 00m 38s) [production]
2016-03-11 §
23:21 <krinkle@tin> Synchronized wmf-config/InitialiseSettings.php: test2wiki favicon (duration: 00m 30s) [production]
22:50 <aaron@tin> Synchronized wmf-config/jobqueue-eqiad.php: Disable persistent redis connections and bump timeout a bit (duration: 00m 38s) [production]
19:40 <jynus> restarting dbstore1002 [production]
18:51 <mutante> mw1001 - removed font again, mw1153 confirmed puppet installs it (only) on imagescalers [production]
18:42 <mutante> mw1001 install fonts-gujr-extra to confirm gerrit 276501 is fine [production]
17:38 <elukey> Increased mediawiki::jobrunner::runners_basic to 30 for mw116[123789] [production]
16:38 <mattflaschen@tin> Synchronized wmf-config/InitialiseSettings.php: Comment-only change (duration: 00m 41s) [production]
15:44 <chasemp> labtestmetal and labtestvirt I am commondeering for some nfs and storage testing that cannot be done in labs, I will reimage when done [production]
15:41 <elukey> forced puppet agent -tv on mw1164 [production]
15:39 <elukey> increased mediawiki::jobrunner::runners_basic from 20 to 30 for mw116[3-6] [production]
15:26 <gehel> deploying https://gerrit.wikimedia.org/r/#/c/274382/ - new defined type not used anywhere, should have less than zero impact. [production]
15:14 <demon@tin> Synchronized multiversion/updateBranchPointers: (no message) (duration: 00m 25s) [production]
15:12 <demon@tin> Synchronized wmf-config/throttle.php: Procomuns Viquimarató - Barcelona throttle rule (duration: 00m 31s) [production]
15:06 <moritzm> installing security updates for openssl, curl, gcrypt, libpng, jasper, expat and libxml2 on mw1018-1025 (and restarts of HHVM) [production]
15:04 <demon@tin> Synchronized README: no-op, co-master sync (duration: 00m 30s) [production]
15:03 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: moving large.dblist back to wmf.16, did not help [production]
14:45 <demon@tin> rebuilt wikiversions.php and synchronized wikiversions files: moving largest.dblist back to wmf.15 for now [production]
14:33 <godog> run swiftrepl thumbs eqiad -> codfw with concurrency 96 [production]
14:25 <godog> run swiftrepl thumbs eqiad -> codfw with concurrency 128 [production]
14:12 <moritzm> installed security updates for openssl, curl, gcrypt, libpng, jasper, expat and libxml2 on mw1017 (other canaries will be upgraded later on if all is well, the rest of mw* on Monday) [production]
13:50 <apergos> on fluorine truncated archive/redis.log-20160311 and archive/JobQueueFederated.log-20160311 to 5mb each, they were each about 500gb [production]
13:26 <mobrovac> restbase deploy end of 3bedb8f5c42 [production]
12:56 <mobrovac> restbase deploy start of 3bedb8f5c42 on canary restbase1001 [production]
12:36 <mobrovac> mathoid deploying 7a282a4181a4 [production]
12:10 <moritzm> uploaded backport of linux-tools 4.4-4 for jessie-wikimedia to carbon (provides kbuild and perf amonst others) [production]
11:48 <akosiaris> restarting gerrit on ytterbium [production]
11:27 <jynus> running authdns-update to solve a dns typo [production]
11:25 <jynus> killed some long-running git-upload-pack tasks [production]
10:26 <hashar> Gerrit task management documented on https://wikitech.wikimedia.org/wiki/Gerrit#Tasks_management [production]
10:11 <hashar> Gerrit: killed stuck "git-upload-pack '/mediawiki/core.git'" tasks [production]
09:46 <elukey> Rebooting rdb200[56].codfw for kernel upgrade [production]
09:13 <ema> umounted /sys/kernel/debug/tracing on cp1067 [production]
09:12 <hashar> Enabling puppet on gallium.wikimedia.org . Been disabled since ~ Mon Mar 7 10:26:01 UTC 2016 [production]
09:07 <moritzm> uploaded linux-meta 1.9 for jessie-wikimedia to carbon (which now defaults linux-meta to installing Linux 4.4) [production]
08:49 <ema> cp1067: apt-get removed linux-image-3.16.0-4-amd64 and linux-image-4.4.0-1-amd64-dbg to free up some disk space [production]
08:20 <moritzm> restarted hhvm on mw1246, mw1258 [production]