|
2012-03-13
§
|
| 18:10 |
<maplebed> |
failover successful, restarted pybal on lvs4, failback successful. |
[production] |
| 18:10 |
<binasher> |
power cycling db1020, which also froze this morning |
[production] |
| 18:08 |
<maplebed> |
stopping pybal on lvs4 - should fail over to lvs3 |
[production] |
| 17:47 |
<maplebed> |
pybal restarted on lvs3 |
[production] |
| 17:47 |
<binasher> |
power cycling db1040, crashed again |
[production] |
| 17:30 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Bug 35183 - p include extensions/Renameuser/Renameuser.php instead of extensions/Renameuser/SpecialRenameuser.php' |
[production] |
| 17:12 |
<mark> |
Sending all normally-pmtpa upload traffic to upload-lb.eqiad |
[production] |
| 17:05 |
<preilly> |
synchronized php-1.19/extensions/ZeroRatedMobileAccess/ZeroRatedMobileAccess.body.php 'changes for zero' |
[production] |
| 16:59 |
<preilly> |
add disable images support to mswiki under zero domain |
[production] |
| 16:59 |
<preilly> |
synchronized wmf-config/CommonSettings.php 'add disable images option for mswiki on zero domain' |
[production] |
| 16:59 |
<preilly> |
synchronized wmf-config/InitialiseSettings.php 'add disable images option for mswiki on zero domain' |
[production] |
| 16:46 |
<preilly> |
synchronized wmf-config/InitialiseSettings.php 'add ZeroRatedMobileAccess extension to mswiki remove from mywiki' |
[production] |
| 16:44 |
<mark> |
Sending traffic from Japan, India, Mexico to upload-lb.eqiad |
[production] |
| 16:37 |
<LeslieCarr> |
reinstalling neon |
[production] |
| 16:23 |
<apergos> |
stole some free space from the phys volume on ms1002 to give us more time for the rsync to keep going til after the move to swift etc |
[production] |
| 15:28 |
<mark> |
Sending traffic from the USA to upload-lb.eqiad |
[production] |
| 15:27 |
<mark> |
Rebooting lvs1005 with upgraded kernel/packages |
[production] |
| 15:12 |
<LeslieCarr> |
manually deleted cp1025 info from nagios config file - nagios restored for now |
[production] |
| 14:51 |
<mark> |
Sending traffic from Canada to upload-lb.eqiad |
[production] |
| 14:33 |
<mark> |
Sending traffic from Brazil to upload-lb.eqiad |
[production] |
| 13:58 |
<mark> |
Sending traffic from Argentina to upload-lb.eqiad |
[production] |
| 12:58 |
<mark> |
Seeding the eqiad upload caches from live upload requests |
[production] |
| 11:59 |
<mark> |
Setup squid logging to oxygen, with oxygen relaying to multicast 233.58.59.1 |
[production] |
| 11:02 |
<mark> |
Rebooting lvs1002 with kernel updates |
[production] |
| 10:17 |
<mark> |
Rebooting manutius with newer 2.6.36 kernel to attempt avoiding i/o kernel bug with torrus |
[production] |
| 02:18 |
<LocalisationUpdate> |
completed (1.19) at Tue Mar 13 02:18:03 UTC 2012 |
[production] |
|
2012-03-12
§
|
| 22:55 |
<K4-713> |
synchronized payments cluster to [[rev:113679|r113679]], and tweaked the anti-fraud rules |
[production] |
| 21:51 |
<catrope> |
synchronized php-1.19/extensions/ArticleFeedbackv5/modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js '[[rev:113671|r113671]]' |
[production] |
| 21:51 |
<catrope> |
synchronized php-1.19/extensions/ArticleFeedbackv5/ArticleFeedbackv5.hooks.php '[[rev:113671|r113671]]' |
[production] |
| 21:44 |
<Reedy> |
Running foreachwiki extensions/WikimediaMaintenance/cleanupBug31576.php in screen as me on hume |
[production] |
| 21:39 |
<RobH> |
search1014 repaired per rt 2483 |
[production] |
| 20:26 |
<RobH> |
cp1040 coming down for hardware stuffs |
[production] |
| 18:19 |
<Nikerabbit> |
Assuming scap has finished |
[production] |
| 17:48 |
<nikerabbit> |
synchronizing Wikimedia installation... : Deploying updated Translate |
[production] |
| 17:46 |
<notpeter> |
restarting indexer on searchidx2 |
[production] |
| 17:24 |
<nikerabbit> |
synchronized php-1.19/includes/Title.php '[[rev:113635|r113635]]' |
[production] |
| 17:22 |
<nikerabbit> |
synchronized php-1.19/languages/ '[[rev:113635|r113635]]' |
[production] |
| 17:14 |
<nikerabbit> |
synchronized php-1.19/extensions/Narayam/ 'Updating Narayam' |
[production] |
| 17:13 |
<mark> |
PXE booting cp1025-cp1028 |
[production] |
| 17:11 |
<nikerabbit> |
synchronized php-1.19/extensions/WebFonts/ 'Updating WebFonts' |
[production] |
| 15:16 |
<mark> |
Rebooted manutius, stuck in a similar state as streber always did |
[production] |
| 06:10 |
<mutante> |
turning off debug mode in nagios-nrpe, again had to kill it , restart fails |
[production] |
| 05:53 |
<mutante> |
dunno, copper was stuck (no mgmt output after reboot) but powercycling it and back |
[production] |
| 05:43 |
<mutante> |
rebooting copper to make sure grub update didnt break it and asked for restart anyways |
[production] |
| 05:37 |
<mutante> |
copper - installing (security) updates (apt,grub,openssl,ruby,libc6..) |
[production] |
| 03:40 |
<mutante> |
same (and nscd) on fenari |
[production] |
| 03:35 |
<mutante> |
upgrading libc6 and related packages on spence |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.19) at Mon Mar 12 02:17:28 UTC 2012 |
[production] |