|
2012-06-28
§
|
| 18:23 |
<RobHalsell> |
swapped bad psu out of ms1001-array3, redundant so no downtime |
[production] |
| 15:40 |
<RobHalsell> |
pulling the following servers, relocating to payments rack: payments1001-1004, boron, beryllium, lithium |
[production] |
| 15:34 |
<RobHalsell> |
dns updated |
[production] |
| 15:31 |
<RobHalsell> |
boron appears to be unallocated, pulling IP allocation, rack allocation, moving to payments per 1227 |
[production] |
| 14:40 |
<mutante> |
svn server is rebooting.brb |
[production] |
| 14:38 |
<mutante> |
dist-upgrading formey (svn/gerrit), rebooting soon |
[production] |
| 14:38 |
<Jeff_Green> |
manganese rebooted for kernel update |
[production] |
| 14:38 |
<RobH> |
allocating yttrium to payments rack per rt 1227 |
[production] |
| 14:25 |
<Jeff_Green> |
manganese dist-upgrade |
[production] |
| 14:15 |
<Ryan_Lane> |
restarting apache on manganese |
[production] |
| 14:15 |
<Ryan_Lane> |
restarting gerrit |
[production] |
| 05:08 |
<Tim> |
srv266 was flooding the fatal error log, complaining about a missing file. Killed apache and ran sync-common. |
[production] |
| 05:03 |
<Tim> |
fixed fatal.log on fenari, socat was writing to a deleted file |
[production] |
| 02:58 |
<LocalisationUpdate> |
completed (1.20wmf5) at Thu Jun 28 02:58:42 UTC 2012 |
[production] |
| 02:30 |
<LocalisationUpdate> |
completed (1.20wmf6) at Thu Jun 28 02:30:03 UTC 2012 |
[production] |
|
2012-06-27
§
|
| 23:50 |
<K4-7131> |
sync'd payments cluster to 592e0a5ba195 |
[production] |
| 23:43 |
<preilly> |
synchronized wmf-config/InitialiseSettings.php 'add rule for mediawiki' |
[production] |
| 22:37 |
<K4-7131> |
sync'd payments cluster to 7e9072c2d571c |
[production] |
| 22:23 |
<binasher> |
temporarily pulling srv211 from pybal |
[production] |
| 21:56 |
<RobH> |
mw1102 has no nic0, rather than troubleshoot it for a long time, reinstall! (rt 3058) |
[production] |
| 21:01 |
<preilly> |
synchronized php-1.20wmf6/extensions/MobileFrontend 'fix CSRF' |
[production] |
| 21:00 |
<preilly> |
synchronized php-1.20wmf5/extensions/MobileFrontend 'fix CSRF' |
[production] |
| 20:55 |
<RobH> |
db1003 back online, replaced mgmt cable and mgmt is working now as well |
[production] |
| 20:51 |
<LeslieCarr> |
rebooting srv266 as it is unresponsive |
[production] |
| 20:44 |
<RobH> |
db1003 mgmt issue due to bad cable, system booting back up, replacing mgmt cable |
[production] |
| 20:35 |
<RobH> |
clean mysql shutdown, db1003 now offline |
[production] |
| 20:33 |
<RobH> |
db1003 mgmt is not responsible, I need to remove power and reboot. confirmed iwth asher this is an s3 slave and can do a short downtime without issues |
[production] |
| 20:32 |
<maxsem> |
synchronized php-1.20wmf5/extensions/MobileFrontend/ 'MF fixes and logging' |
[production] |
| 20:29 |
<maxsem> |
synchronized php-1.20wmf6/extensions/MobileFrontend/ 'MF fixes and logging' |
[production] |
| 19:56 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'disable LastModified and LastModified/E3Experiment' |
[production] |
| 19:54 |
<reedy> |
synchronized php-1.20wmf6/maintenance/runJobs.php |
[production] |
| 19:53 |
<reedy> |
synchronized php-1.20wmf5/maintenance/runJobs.php |
[production] |
| 19:32 |
<reedy> |
synchronized php-1.20wmf6/extensions/Translate/specials/SpecialLanguageStats.php |
[production] |
| 19:19 |
<reedy> |
synchronized php-1.20wmf6/extensions/Translate/ |
[production] |
| 19:06 |
<reedy> |
synchronized php-1.20wmf6/extensions/Translate/ |
[production] |
| 19:03 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: meta back to wmf6, not cause of translate issues |
[production] |
| 18:53 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: closed to 1.20wmf6 |
[production] |
| 18:51 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikimedia wikis to 1.20wmf6 |
[production] |
| 18:48 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wiktionary and wikiversity to 1.20wmf6 |
[production] |
| 18:47 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikisource and wikiquote to 1.20wmf6 |
[production] |
| 18:47 |
<Jeff_Green> |
added several mobile hostnames to DNS for RT #2996 |
[production] |
| 18:46 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikinews and wikibooks to 1.20wmf6 |
[production] |
| 18:44 |
<aaron> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Moved metawiki back to 1.20wmf5 |
[production] |
| 18:41 |
<K4-713> |
synchronized payments cluster to fundraising/1.20 de0256084a |
[production] |
| 18:33 |
<aaron> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Moved special wikis to php-1.20wmf6 |
[production] |
| 18:29 |
<aaron> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Moved en(wikibooks|wikinews|wikiquote|wikisource|wikiversity|wiktionary) to 1.20wmf6 |
[production] |
| 18:05 |
<RobH> |
cp1017 memory replaced |
[production] |
| 17:52 |
<RobH> |
cp1017 is offline due to memory error. replacement memory on site, pulling system for swap |
[production] |
| 17:46 |
<preilly> |
synchronized php-1.20wmf6/extensions/ZeroRatedMobileAccess 'update for landing page' |
[production] |
| 17:46 |
<asher> |
synchronized wmf-config/mc.php 'disabling wgMemCachedPersistent; lowering wgMemCachedTimeout to 2x client default from 30x default' |
[production] |