2301-2350 of 10000 results (21ms)
2012-12-16 §
02:40 <LeslieCarr> checking memcache servers due to spate of mediawiki servers flapping [production]
02:26 <LocalisationUpdate> completed (1.21wmf6) at Sun Dec 16 02:26:23 UTC 2012 [production]
2012-12-15 §
14:37 <maxsem> synchronized /php-1.21wmf5/includes/Message.php 'https://gerrit.wikimedia.org/r/#/c/38822/' [production]
14:30 <paravoid> powercycling ms-be1 [production]
12:18 <apergos> ms-be1 can't be reached form mgmt console to check on it (Connection refused) [production]
02:47 <LocalisationUpdate> completed (1.21wmf5) at Sat Dec 15 02:47:17 UTC 2012 [production]
02:45 <LeslieCarr> drained all transit traffic from cr1-eqiad -- restore by load override /var/home/lcarr/cr1-eqiad.undrained if traffic levels seem too high [production]
02:25 <LocalisationUpdate> completed (1.21wmf6) at Sat Dec 15 02:25:26 UTC 2012 [production]
00:36 <LeslieCarr> appears that cr1-eqiad's fpc rebooted [production]
2012-12-14 §
23:25 <AaronSchulz> Compiled texvc into uncommon/php-1.21wmf6 dir for all apaches [production]
23:08 <K4-713> synchronized payments to 4f5296e43b1c708 [production]
22:34 <mutante> fixing symlink on kaulen to reflect new bugzilla version 4.2.4 [production]
22:30 <hashar> restarting Zuul on gallium [production]
20:54 <mutante> deleting /var/lib/git/integration on gallium [production]
18:54 <mutante> appyling mini patch on bugzilla per http://bzr.mozilla.org/bugzilla/4.2/revision/8176 [production]
18:31 <reedy> synchronized wmf-config/InitialiseSettings.php 'Enable clicktracking on test2wiki' [production]
16:56 <sbernardin> sq48 going down for HDD controller card replacement-rt4041 [production]
15:40 <demon> synchronized wmf-config/throttle.php 'Deploying I330230e6' [production]
09:30 <mlitn> synchronized php-1.21wmf5/extensions/ArticleFeedbackv5 'Fix AFTv5 toolbox popups' [production]
02:47 <LocalisationUpdate> completed (1.21wmf5) at Fri Dec 14 02:46:58 UTC 2012 [production]
02:26 <LocalisationUpdate> completed (1.21wmf6) at Fri Dec 14 02:25:58 UTC 2012 [production]
01:50 <mutante> restarting pdns on ns0 [production]
01:35 <mutante> creating new self-signed SSL cert for wikitech - valid for another year [production]
01:03 <csteipp> synchronized php-1.21wmf6/extensions/CategoryTree/CategoryTreeFunctions.php [production]
00:15 <olivneh> synchronized php-1.21wmf6/extensions/AbuseFilter [production]
00:15 <mutante> prototype.wikimedia is also removed ... bye or please complain to us about it [production]
00:13 <olivneh> synchronized php-1.21wmf5/extensions/AbuseFilter [production]
00:09 <mutante> DNS update - killing a bunch of .tesla. [production]
2012-12-13 §
23:55 <olivneh> synchronized wmf-config/InitialiseSettings.php 'Enabling GettingStarted on enwiki' [production]
23:42 <olivneh> synchronized wmf-config/abusefilter.php '([[bugzilla:42758|bug 42758]]) enable AbuseFilter events for enwiki irc feed' [production]
23:40 <mutante> gallium mysql (jenkins/CI) back up and running after brief db server issue [production]
23:34 <olivneh> synchronized php-1.21wmf6/extensions/AbuseFilter [production]
23:33 <olivneh> synchronized php-1.21wmf5/extensions/AbuseFilter [production]
23:30 <mutante> mysql stopped on gallium due to dpkg issue.. investigating [production]
23:28 <mutante> installing package upgrades on gallium, incl. newer mysql packages [production]
23:20 <olivneh> synchronized php-1.21wmf6/extensions/E3Experiments/experiments/acux/accountCreationUX.js [production]
23:17 <olivneh> synchronized php-1.21wmf5/extensions/E3Experiments/experiments/acux/accountCreationUX.js [production]
23:14 <spage> Finished syncing Wikimedia installation... : E3 deploy new Special:GettingStarted and E3 logging [production]
23:04 <LeslieCarr> switched asw-c-eqiad to cr2-eqiad aggregated bundle from lacp to static [production]
22:34 <spage> Started syncing Wikimedia installation... : E3 deploy new Special:GettingStarted and E3 logging [production]
20:10 <mlitn> synchronized php-1.21wmf5/extensions/ArticleFeedbackv5 'Update ArticleFeedbackv5 to master' [production]
19:39 <notpeter> restarting all lucene instances in pmtpa [production]
19:27 <notpeter> stopping puppet on brewster [production]
19:01 <notpeter> upgrading lucene to 2.1.6wm1 on all search boxes in pmtpa [production]
18:55 <notpeter> upgrading lucene to 2.1.6wm1 on all search boxes in eqiad [production]
18:51 <notpeter> taking down hume for disk replacement/reimage/etc [production]
18:51 <paravoid> nas1-a/nas1001-a: resizing originals volume to 37T and thumbs volume to 6T [production]
16:54 <nikerabbit> synchronized php-1.21wmf6/extensions/Translate/ 'Translate bugfix' [production]
14:08 <^demon> live-hacked out manganese -> gallium gerrit replication. spamming logs with "reject HostKey: gallium.wikimedia.org". restarted gerrit on manganese. [production]
13:53 <MaxSem> Updated enwiki Solr index [production]