2010-02-23
§
|
23:41 |
<domas> |
stopped nginx on en.m until we resolve special:random issue |
[production] |
20:53 |
<root> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Bug 22628 - Logo of Chamorro Wikipedia' |
[production] |
17:23 |
<Rob> |
drac setup on new servers ersch, formey, alsted, and kaulen. still need port allocation. |
[production] |
16:47 |
<Rob> |
forgot to add mgmt, updating dns |
[production] |
16:43 |
<Rob> |
updating dns for new servers ersch, formey, alsted, & kaulen |
[production] |
15:37 |
<Rob> |
FINALLY got in the last RMA drive for storage1, arrays were toast (losing 1/2 the drives in a raid10 and having the unlucky chance of them being paired), reinstall in progress |
[production] |
12:50 |
<RoanKattouw> |
Cleaned up my dblist voodoo, everything should be back to normal now |
[production] |
12:41 |
<RoanKattouw> |
Fixed sync-dblist, was broken |
[production] |
12:31 |
<RoanKattouw> |
Syncing pmtpa.dblist anyway, seems I have to |
[production] |
12:24 |
<RoanKattouw> |
Temporarily adding wikis listed in bug 20325 to pmtpa.dblist so I can export their contents. Please don't sync this |
[production] |
2010-02-22
§
|
23:40 |
<Tim> |
restarted replication on ms2 |
[production] |
23:39 |
<Tim> |
ms1 is up with a full copy of ms3 data, now replicating cluster22/rc1 |
[production] |
21:09 |
<mark> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Revert' |
[production] |
20:30 |
<root> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Bug 22522 - Allow bureaucrats to remove sysop flag on Simple English Wikitionary' |
[production] |
20:09 |
<Rob> |
finished the setup of srv217 (just needed puppet fixed and files updated) |
[production] |
20:02 |
<root> |
ran sync-common-all |
[production] |
20:01 |
<mark> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Set vector as default skin on nlwikimedia' |
[production] |
19:44 |
<root> |
ran sync-common-all |
[production] |
18:10 |
<Fred> |
ganglia is going to hiccup a little for the next few minutes while I restore data... "Don't panic" |
[production] |
07:53 |
<Tim> |
rebooted srv206 via drac |
[production] |
07:48 |
<tstarling> |
synchronized php-1.5/wmf-config/mc.php 'removing srv206, is down' |
[production] |
06:19 |
<Tim> |
increased multicast TTL from 1 to 3 in gmond_template.erb, to fix ganglia on ms* |
[production] |
05:54 |
<Tim> |
started ms2 -> ms1 copy using nc+tarpipe |
[production] |
05:30 |
<Tim> |
shutting down mysql on ms2 for copy to ms1 |
[production] |
05:09 |
<tstarling> |
synchronized php-1.5/wmf-config/db.php 'depooling ms2 for copy to ms1' |
[production] |