|
2011-07-27
§
|
| 23:47 |
<preilly> |
pushing bad redirect logging to mobile cluster |
[production] |
| 23:37 |
<maplebed> |
started reslave from db42 to db1047; poor performance expected for the duration of the sync. |
[production] |
| 23:33 |
<notpeter> |
restarting torrus again |
[production] |
| 23:19 |
<preilly> |
pushing badredirect log to mobile cluster |
[production] |
| 22:46 |
<notpeter> |
upgrading squid and squid-frontend on knsq27 and knsq28 |
[production] |
| 22:13 |
<notpeter> |
torrus stuck. assuming in deadlock. following instructions for restart |
[production] |
| 22:01 |
<AaronSchulz> |
Re-installed moodbar & edittracking. Cleaned up wmf-config so that we are on HEAD (but with the reverted stuff still gone) |
[production] |
| 22:00 |
<aaron> |
synchronizing Wikimedia installation... Revision: 93132: |
[production] |
| 20:19 |
<notpeter> |
upgrading squid and squid-frontend on knsq29 and knsq30 |
[production] |
| 19:54 |
<binasher> |
pulled sq78 frontend from lvs |
[production] |
| 19:40 |
<Reedy> |
Reports of users using non mobile browsers being redirected to mobile sites (enwiki and dewiki) |
[production] |
| 19:04 |
<mark> |
Temporarily disabled facilities.pp file in puppet, until there is time to debug |
[production] |
| 18:35 |
<binasher> |
pulled sq78 |
[production] |
| 18:32 |
<binasher> |
sq78 frontend is in service with mobile redirection |
[production] |
| 17:08 |
<binasher> |
pulling sq78 from lvs prior to mobile testing |
[production] |
| 16:56 |
<aaron> |
synchronized multiversion/MWMultiVersion.php 'deployed [[rev:93255|r93255]]' |
[production] |
| 13:40 |
<Reedy> |
Do not push any configuration change from wmf-config without speaking to Reedy/Ariel first |
[production] |
| 13:33 |
<Reedy> |
Seems some configuration files between [[rev:2087|r2087]] and [[rev:2088|r2088]] weren't pushed to site, pushing them broke most projects. Aaron to investigate when he's in the office |
[production] |
| 13:20 |
<reedy> |
synchronized wmf-config/extension-list '[[rev:2087|r2087]]' |
[production] |
| 13:19 |
<reedy> |
synchronized wmf-config/liquidthreads.php '[[rev:2087|r2087]]' |
[production] |
| 13:19 |
<reedy> |
synchronized wmf-config/ExtensionMessages.php '[[rev:2087|r2087]]' |
[production] |
| 13:18 |
<Reedy> |
Site should be somewhat back to normal, reverted to [[rev:2087|r2087]] of config from private SVN |
[production] |
| 13:17 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php '[[rev:2087|r2087]]' |
[production] |
| 13:17 |
<reedy> |
synchronized wmf-config/CommonSettings.php '[[rev:2087|r2087]]' |
[production] |
| 13:15 |
<reedy> |
synchronized wmf-config/CommonSettings-old.php |
[production] |
| 13:14 |
<reedy> |
synchronized wmf-config/CommonSettings-new.php |
[production] |
| 13:12 |
<reedy> |
synchronized wmf-config/CommonSettings.php '[[rev:2088|r2088]]' |
[production] |
| 13:12 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php '[[rev:2088|r2088]]' |
[production] |
| 12:36 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Reverting last push' |
[production] |
| 12:36 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Reverting last push' |
[production] |
| 12:32 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Reverting last push' |
[production] |
| 12:31 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Reverting last push' |
[production] |
| 12:27 |
<andrewS> |
miguel gave OK to fill up d1 in sdtpa as the heat flow issues have been temporarily resolved (also said construction on new AC units was to begin Aug 1st but has not heard anything yet). |
[production] |
| 12:20 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 12:20 |
<reedy> |
synchronized wmf-config/CommonSettings.php |
[production] |
| 11:55 |
<mutante> |
sq63 - restored software RAID & dist-upgrade & reboot |
[production] |
| 11:15 |
<Jeff_Green> |
adjusted firewall rules on payments servers per RT #1223 |
[production] |
| 08:55 |
<andrewS> |
74th [[rev:410|r410]] dell server has arrived in tampa. uploaded packing slip to rt#611. |
[production] |
| 06:13 |
<Jeff_Green> |
repooling payments3 & payments4, they're dist-updated and tested |
[production] |
| 05:05 |
<aaron> |
synchronized php/maintenance/jobs-loop.sh 'added --wiki to another spot' |
[production] |
| 02:49 |
<aaron> |
synchronized wmf-config/CommonSettings.php 'fixed texvc path' |
[production] |
| 01:31 |
<notpeter> |
restarting jorb runners on srv151-180. also wishing there was a script for htis |
[production] |
| 01:05 |
<aaron> |
synchronized php/maintenance/jobs-loop.sh 'added wiki=aawiki param to nextJobDB.php' |
[production] |
| 00:54 |
<^demon> |
job runners stuck in futex deadlock again :( |
[production] |