1801-1850 of 10000 results (12ms)
2012-07-01 §
01:43 <notpeter> that worked. restarting all remaining search nodes. [production]
01:39 <notpeter> problem with lucene persisting through service restart, but not node restart. restarting en pool nodes. [production]
01:20 <paravoid> restarting opendj (nfs1/nfs2), load spike, possibly related to leap second [production]
00:51 <notpeter> search1004 dead. powercycling. [production]
00:50 <notpeter> based on ganglia evidence, lucene seems to have been affected by leap second bug. restartig each instance, one minute wait in between [production]
2012-06-30 §
16:12 <mark> Temporarily added path 6939+ 14907+ to AVOID-PATHs on cr2-knams [production]
02:53 <LocalisationUpdate> completed (1.20wmf5) at Sat Jun 30 02:53:46 UTC 2012 [production]
02:28 <maplebed> corrected LVS pdns_recursor config error causing DNS queries to fail on LVS servers in gerrit [[rev:13554|r13554]] and [[rev:13555|r13555]]. [production]
02:27 <LocalisationUpdate> completed (1.20wmf6) at Sat Jun 30 02:27:08 UTC 2012 [production]
2012-06-29 §
19:49 <hashar> restarting Jenkins to fix an issue with "parameterized builds" plugin. Updated git plugin as well. [production]
19:35 <RobH> dns update via authdns-update for vanadium ip [production]
18:05 <Jeff_Green> sync-apache and apache-graceful-all for http://donate.wikimedia.org-->https redirect [production]
16:02 <RobH> ms-be1001 and ms-be1002 powering down for ssd installation [production]
15:59 <RobH> authdns-update run [production]
15:47 <RobH> updating dns [production]
15:16 <mutante> dist-upgrading srv280,srv270,srv264 [production]
15:11 <Jeff_Green> apache-graceful-all for redirect conf change [production]
15:10 <Jeff_Green> sync-apache to push out new foundation.conf [production]
14:50 <mark> Reinstalled chromium with precise [production]
13:46 <hashar> fixed interwiki on http://wikisource.org/ main page by hacking a script in production and refreshing cache [production]
13:46 <hashar> synchronized php-1.20wmf6/cache/interwiki.cdb 'Updating interwiki cache for 1.20wmf6' [production]
13:32 <hashar> synchronized php/cache/interwiki.cdb 'Updating interwiki cache' [production]
12:38 <dzahn> synchronized php/cache/interwiki.cdb 'Updating interwiki cache' [production]
12:38 <mutante> dumping interwiki and updating interwiki cache (to fix broken interwiki links, like wikisource.org -> wikipedia.org) [production]
09:31 <hashar> Jenkins: deployed gitsqlhaschanged patch ( d04f779 0f069c3 integration/jenkins.git ) [production]
07:56 <hashar> synchronized wmf-config/CommonSettings.php 'send header from CS.php only for non CLI scripts {{gerrit|13435}}' [production]
07:08 <mutante> upgrading apt packages on brewster [production]
02:51 <LocalisationUpdate> completed (1.20wmf5) at Fri Jun 29 02:51:13 UTC 2012 [production]
02:26 <LocalisationUpdate> completed (1.20wmf6) at Fri Jun 29 02:26:25 UTC 2012 [production]
2012-06-28 §
23:30 <reedy> synchronized php-1.20wmf6/includes/resourceloader/ResourceLoader.php [production]
23:15 <binasher> completed aft offload_large_feedback migration on enwiki [production]
23:03 <asher> synchronized wmf-config/db.php 'returning db36' [production]
21:50 <kaldari> Finished syncing Wikimedia installation... : [production]
21:39 <kaldari> Started syncing Wikimedia installation... : [production]
21:13 <asher> synchronized wmf-config/db.php 'temp pulling db36' [production]
20:38 <binasher> ran aftv5 offload_large_feedback migrations on testwiki and en_labswikimedia [production]
20:14 <RobH> dns update for pc1-pc3 [production]
19:54 <kaldari> Finished syncing Wikimedia installation... : [production]
19:08 <kaldari> Started syncing Wikimedia installation... : [production]
18:35 <hashar> so the nicely reviewed changes broke the enwiki stylesheets :/ reverted change :-((( [production]
18:34 <hashar> synchronized wmf-config/CommonSettings.php [production]
18:33 <hashar> srv190 and srv281 got ssh timeout [production]
18:31 <hashar> synchronized wmf-config/CommonSettings.php [production]
18:30 <hashar> did various tests using eval.php. Most important is $realm -> production. $cluster -> pmtpa. Syncing [production]
18:25 <hashar> updating mediawiki-config to grab a12545d edceb4c & eee97ad [production]
18:23 <RobHalsell> swapped bad psu out of ms1001-array3, redundant so no downtime [production]
15:40 <RobHalsell> pulling the following servers, relocating to payments rack: payments1001-1004, boron, beryllium, lithium [production]
15:34 <RobHalsell> dns updated [production]
15:31 <RobHalsell> boron appears to be unallocated, pulling IP allocation, rack allocation, moving to payments per 1227 [production]
14:40 <mutante> svn server is rebooting.brb [production]