7101-7150 of 10000 results (47ms)
2015-05-06 §
17:03 <jgage> hadoop active namenode switched back to analytics1001 after rack C4 switch replacement [production]
16:43 <apergos> done with all trusty salt updates in pro except for labcontrol1002 (?), doing jessie now in very tiny batches, it's being trouble [production]
15:29 <bd808> Stashed uncommitted change to scap on tin that disabled php opening tag check for sync-file [production]
15:27 <bd808> Updated scap to 57036d2 (Update statsd events) [production]
15:27 <bd808> trebuchet checkout for scap/scap failed for mw1113.eqiad.wmnet, mw1222.eqiad.wmnet, mw1104.eqiad.wmnet [production]
15:25 <bd808> trebuchet fetch for scap/scap failed on mw1222.eqiad.wmnet [production]
15:04 <bd808> Synchronized wmf-config/InitialiseSettings.php: Send group0 + group1 MediaWiki events to logstash {{gerrit|209170}} (duration: 00m 16s) [production]
14:32 <cmjohnson1> shutting down db1054 for maintenance [production]
14:22 <_joe_> depooling the HHVM imagescaler [production]
14:20 <Nemo_bis> phabricator went down again for some minutes, seems ok now? [production]
14:17 <_joe_> pooling the HHVM imagescalers to test if the issues are solved now. [production]
14:15 <andrewbogott> rebooting labvirt1009 one last time [production]
13:53 <_joe_> upgrading the hhvm imagescaler (mw1152) to HHVM 3.6.1 [production]
13:47 <springle> Synchronized wmf-config/db-eqiad.php: repool db1021 in s2, warm up (duration: 00m 27s) [production]
13:42 <apergos> all precise hosts are upgraded to salt except for tin and virt1000; in the middle of trusty updates now, in batches [production]
13:38 <_joe_> uploading HHVM 3.6.1 and all the related extensions to apt.wikimedia.org [production]
13:01 <paravoid> replacing asw-c4-eqiad (T93730) [production]
12:45 <krenair> Synchronized php-1.26wmf4/extensions/SemanticMediaWiki/specials/QueryPages/SMW_QueryPage.php: https://gerrit.wikimedia.org/r/#/c/209212/ (duration: 00m 21s) [production]
08:12 <legoktm> Synchronized wmf-config/CommonSettings-labs.php: no-op (duration: 00m 24s) [production]
07:20 <ori> Synchronized wmf-config/CommonSettings.php: I019944f42: Change EventLogging endpoint to /beacon/event (duration: 00m 14s) [production]
06:51 <LocalisationUpdate> ResourceLoader cache refresh completed at Wed May 6 06:50:27 UTC 2015 (duration 50m 26s) [production]
03:14 <LocalisationUpdate> completed (1.26wmf4) at 2015-05-06 03:13:28+00:00 [production]
03:09 <l10nupdate> Synchronized php-1.26wmf4/cache/l10n: (no message) (duration: 08m 46s) [production]
02:46 <LocalisationUpdate> completed (1.26wmf3) at 2015-05-06 02:45:26+00:00 [production]
02:36 <l10nupdate> Synchronized php-1.26wmf3/cache/l10n: (no message) (duration: 10m 46s) [production]
02:27 <springle> xtrabackup clone db1060 to db1021 [production]
02:04 <ori> Synchronized wmf-config/CommonSettings.php: I83ad6d060: Remove wmgUseBits setting, now that the migration is complete (duration: 00m 18s) [production]
02:02 <aude> Synchronized php-1.26wmf4/extensions/Wikidata: Fix Wikibase api error output bug - update submoduled (duration: 00m 28s) [production]
01:59 <aude> Synchronized php-1.26wmf4/extensions/Wikidata: Fix Wikibase api error output bug (duration: 01m 08s) [production]
01:52 <ori> Synchronized multiversion/MWWikiversions.php: Ib08e36901: MWWikiversions::readDbListFile: allow single-line ("#" or "//") comments (duration: 00m 18s) [production]
01:40 <springle> upgrade db1021 trusty [production]
00:51 <springle> schema change running T95179 wikidata, bit unusual, dropping a not-null field [production]
00:46 <bd808> Synchronized wmf-config/CommonSettings.php: Add AffCom user group application contact page on meta {{gerrit|207332}} (duration: 00m 20s) [production]
00:45 <bd808> Synchronized docroot/noc/createTxtFileSymlinks.sh: Add AffCom user group application contact page on meta {{gerrit|207332}} (duration: 00m 17s) [production]
00:45 <bd808> Synchronized docroot/noc/conf/AffComContactPages.php.txt: Add AffCom user group application contact page on meta {{gerrit|207332}} (duration: 00m 15s) [production]
00:44 <bd808> Synchronized wmf-config/AffComContactPages.php: Add AffCom user group application contact page on meta {{gerrit|207332}} (duration: 00m 33s) [production]
00:15 <catrope> Synchronized php-1.26wmf3/extensions/Flow: SWAT (duration: 00m 23s) [production]
00:15 <catrope> Synchronized php-1.26wmf3/extensions/WikiEditor: SWAT (duration: 00m 33s) [production]
00:13 <bd808> Aborted sync-common on snapshot1004; host is starved for RAM and using swap heavily [production]
00:06 <catrope> Synchronized php-1.26wmf4/extensions/CirrusSearch: SWAT (duration: 00m 28s) [production]
00:06 <catrope> Synchronized php-1.26wmf4/extensions/Flow: SWAT (duration: 00m 52s) [production]
00:04 <catrope> Synchronized php-1.26wmf4/extensions/WikiEditor: SWAT (duration: 00m 42s) [production]
2015-05-05 §
23:57 <bd808> aborted and restarted sync-common on snapshot1004.eqiad.wmnet manually after waiting 24 minutes with no progress [production]
23:49 <catrope> Synchronized wmf-config/InitialiseSettings.php: Use Wiki.svg for wikimania2015wiki logo (duration: 00m 19s) [production]
23:47 <jgage> switched hadoop active namenode from analytics1001 to analytics1002 for rack C4 switch replacement tomorrow morning (T93730) [production]
23:39 <rmoen> Finished scap: Updates for Gather and MobileFrontend (duration: 41m 11s) [production]
23:33 <bd808> running sync-common on snapshot1004.eqiad.wmnet manually after it was aborted in scap by rmoen [production]
23:30 <bd808> snapshot1004.eqiad.wmnet hanging scap yet again [production]
23:23 <mutante> deleted 8G recurring_blocked.tsv from lutetium [production]
22:58 <rmoen> Started scap: Updates for Gather and MobileFrontend [production]