|
2012-03-30
§
|
| 12:47 |
<mutante> |
powercycling db1047 |
[production] |
| 12:28 |
<mutante> |
deleted old kernel sources on upgraded srvs for that little extra space during peaks, suggesting to nuke /usr/share/doc if there should be more disk space warnings |
[production] |
| 10:41 |
<mutante> |
same for srv223 |
[production] |
| 09:18 |
<mutante> |
srv224,srv219,srv220, upgrade apache, dist-upgrading w/ kernel, disabling ureadahead, rebooting one by one |
[production] |
| 08:06 |
<mutante> |
storage3 - gmond unable to find the metric information for any mysql_* .."module has not been loaded", starting mysql, running puppet ... |
[production] |
| 07:57 |
<mutante> |
powercycling storage3 |
[production] |
| 07:03 |
<Tim> |
running [[bugzilla:35578|bug 35578]] cleanup script in screen on fenari |
[production] |
| 06:41 |
<tstarling> |
synchronized php-1.19/includes/specials/SpecialBlock.php |
[production] |
| 06:40 |
<tstarling> |
synchronized php-1.19/includes/specials/SpecialBlock.php |
[production] |
| 06:39 |
<tstarling> |
synchronized php-1.19/includes/specials/SpecialBlock.php |
[production] |
| 06:15 |
<Tim> |
killed vi on fenari owned by awjrichards, locking CommonSettings.php for two days |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.19) at Fri Mar 30 02:17:56 UTC 2012 |
[production] |
| 01:13 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Remove more crap' |
[production] |
| 01:02 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Remove some dupe code' |
[production] |
| 01:00 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Remove wmgUsabilityPrefSwitch' |
[production] |
| 00:59 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Remove wmgUsabilityPrefSwitch' |
[production] |
| 00:58 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Remove unused wmgUseUsabilityInitiativeAlpha' |
[production] |
|
2012-03-29
§
|
| 23:49 |
<aaron> |
synchronized php-1.19/includes/revisiondelete/RevisionDeleteUser.php 'deployed [[rev:114619|r114619]]' |
[production] |
| 21:20 |
<LeslieCarr> |
rebooting db47 |
[production] |
| 20:28 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Swap wgUseCommaCount to wgArticleCountMethod' |
[production] |
| 20:07 |
<notpeter> |
restarting lsearchd on search2 to del the logfile to end all logfiles |
[production] |
| 20:05 |
<RoanKattouw> |
Stopping and starting Gerrit on manganese to apply Chad's change of the -1 text in the DB |
[production] |
| 20:02 |
<notpeter> |
restarting lsearchd on search7 to del the logfile to end all logfiles |
[production] |
| 18:11 |
<catrope> |
synchronized php-1.19/extensions/ClickTracking/ClickTracking.hooks.php |
[production] |
| 17:59 |
<RobH> |
search1021 coming back up, done with tests |
[production] |
| 17:53 |
<RobH> |
search1021 coming down for ssd fit test |
[production] |
| 17:07 |
<notpeter> |
disabling notifications for search lvs nagios checks for 24 hours to test fix |
[production] |
| 15:42 |
<notpeter> |
finished clearning up all pmtpa search hosts. hey look! they all have lots of space now! |
[production] |
| 15:15 |
<notpeter> |
restarting lsearchd on search3 |
[production] |
| 15:02 |
<RobH> |
brewster puppet re-enabled |
[production] |
| 15:02 |
<RobH> |
virt1001 pxe boots via dhcp and fails tftp download, i have to hold off on further troubleshooting until i have a network admin |
[production] |
| 14:47 |
<RobH> |
did virt1001 wrong, reupdating dns |
[production] |
| 14:39 |
<RobH> |
all nameservers still online after udpate |
[production] |
| 14:37 |
<RobH> |
updating dns for virt1001 testing |
[production] |
| 14:29 |
<RobH> |
stopping puppet runs on brewster so my hacking at the dhcpd.conf file won't get overwritten until I have it working right |
[production] |
| 14:01 |
<Jeff_Green> |
restarted varnish on on cp3002 because it was thrashing futiley |
[production] |
| 13:45 |
<notpeter> |
rebooting (mostly) down cp3001 |
[production] |
| 13:17 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Add participation namespace to metawiki per request' |
[production] |
| 13:11 |
<notpeter> |
trimming logs and such on search1-20 |
[production] |
| 09:59 |
<mutante> |
srv221, disabling ureadahead, installing package upgrades and new kernel, rebooting |
[production] |
| 09:40 |
<mutante> |
kill and start lsearchd on search7 |
[production] |
| 09:36 |
<mutante> |
restarted defunct lsearchd on search6 |
[production] |
| 09:10 |
<mutante> |
gallium - added demon,hashar,reedy to group jenkins as it's a problem using puppet when users and groups already exist |
[production] |
| 06:25 |
<mutante> |
powercycling sq40 |
[production] |
| 06:21 |
<mutante> |
installed more package upgrades on sodium |
[production] |
| 05:58 |
<mutante> |
installed security upgrades on brewser, cadmium, capella (apache,mysql,ruby,apt..) |
[production] |
| 05:49 |
<mutante> |
db42 - mysql did not autostart after boot, added using update-rc.d |
[production] |
| 05:42 |
<mutante> |
db42 - reboot worked despite the grub warning about unreliable blocklists |
[production] |
| 05:37 |
<mutante> |
rebooting db42 to finish upgrades |
[production] |
| 02:18 |
<LocalisationUpdate> |
completed (1.19) at Thu Mar 29 02:17:53 UTC 2012 |
[production] |