|
2009-10-07
§
|
| 17:41 |
<mark> |
Installed puppet on tridge |
[production] |
| 17:28 |
<brion> |
js2-borne code is breaking wikibits.js again; mdale please make sure $wgStylePath is respected |
[production] |
| 17:22 |
<brion> |
during update some brief borkage with missing stylesheets. inconsistency in file update order? seems ok now |
[production] |
| 16:58 |
<Rob> |
decommissioning srv48, srv49, srv50. removed from server_roles, node groups, running wipe in rack |
[production] |
| 16:52 |
<Rob> |
srv81 crashed, decommissioning, removed from node groups, server roles, lvs, wiping in rack |
[production] |
| 16:35 |
<Rob> |
pulled srv44-srv47 from rack, decommissioned |
[production] |
| 16:17 |
<Rob> |
srv31 already wiped, unracked |
[production] |
| 16:15 |
<mark> |
Installed puppet on singer |
[production] |
| 16:14 |
<Rob> |
srv31 still racked, but is crashed, decommissioning. |
[production] |
| 16:12 |
<mark> |
Installed puppet on streber |
[production] |
| 16:10 |
<Rob> |
decommissioning srv137, pulled from rack, wiped, pulled from nodegroups and lvs |
[production] |
| 16:09 |
<mark> |
Installed puppet on mchenry and sanger |
[production] |
| 16:08 |
<mark> |
Removed puppet again on isidore; it's ubuntu 7.04 and really needs to be replaced soon |
[production] |
| 16:06 |
<mark> |
Installed puppet on isidore |
[production] |
| 16:05 |
<mark> |
Installed puppet on pdf1 |
[production] |
| 16:00 |
<mark> |
Installed puppet on bayle and bayes |
[production] |
| 15:59 |
<mark> |
Installed puppet on db20 |
[production] |
| 15:58 |
<Rob> |
negate that, not depooling. |
[production] |
| 15:58 |
<Rob> |
depooling sq43 until later today when i can poke at it. |
[production] |
| 15:55 |
<Rob> |
srv131 decommissioning, removed from lvs, node groups and server roles, wiping in rack. |
[production] |
| 15:49 |
<mark> |
Installed puppet on spence and williams |
[production] |
| 15:47 |
<mark> |
Installed puppet on lvs2-4 |
[production] |
| 15:42 |
<Rob> |
redeployed sq43 |
[production] |
| 15:41 |
<mark> |
Installed puppet on storage1-3 |
[production] |
| 15:35 |
<Rob> |
srv130 was down, out of warranty, decommissioned, removed from node groups and lvs, wiping in rack |
[production] |
| 15:32 |
<mark> |
Shutdown srv9 for decommissioning |
[production] |
| 15:31 |
<mark> |
Installed puppet on amane and erzurumi |
[production] |
| 15:25 |
<mark> |
Installed puppet on fenari and mobile1 |
[production] |
| 15:20 |
<mark> |
Installed puppet on ms2 and ms3 |
[production] |
| 15:19 |
<Rob> |
srv122 decommissioning, pulled from nodes and lvs, wiping in rack |
[production] |
| 15:10 |
<mark> |
Installed puppet on hume |
[production] |
| 15:10 |
<Rob> |
srv90 pulled, decommissioned, wiped, and out of node groups and nagios |
[production] |
| 15:07 |
<Rob> |
lucene group in nagios sync was halting it, since no hosts exist in that group anymore. commented out group in nagios conf.php and synced. |
[production] |
| 15:00 |
<Rob> |
server rose decommissioning, removed from nodegroups, pulled from rack, removed from server roles |
[production] |
| 14:57 |
<Rob> |
srv118 running wipe, removed network connection. |
[production] |
| 14:56 |
<Rob> |
srv118 down, reboots for no good reason, out of warranty, decommissioning, removed from LVS, nodegroups, wiping in rack |
[production] |
| 14:43 |
<Rob> |
decommissioned coronelli, pulled from rack, removed from server roles, removed from node groups |
[production] |
| 14:39 |
<Rob> |
decommissioned maurus, pulled from rack, removed from server roles, removed from node groups |
[production] |
| 02:21 |
<Tim> |
hard reset of mobile1, was not responding on ssh or serial |
[production] |
| 02:18 |
<brion> |
mobile1 been down for a bit; Tim is poking at remote console to reboot it |
[production] |
| 01:54 |
<brion> |
testing svn up on test |
[production] |