|
2012-01-09
§
|
| 19:43 |
<RobH> |
torrus dead, kicking |
[production] |
| 19:32 |
<nikerabbit> |
synchronized wmf-config/CommonSettings.php 'Updating Translate config 2/2' |
[production] |
| 19:32 |
<nikerabbit> |
synchronized wmf-config/InitialiseSettings.php 'Updating Translate config 1/2' |
[production] |
| 19:16 |
<nikerabbit> |
synchronized php-1.18/includes/parser/Parser.php 'Deploying [[rev:108461|r108461]]' |
[production] |
| 19:07 |
<catrope> |
synchronized php-1.18/extensions/UploadWizard/resources/mw.UploadWizardLicenseInput.js '[[rev:108459|r108459]]' |
[production] |
| 18:55 |
<nikerabbit> |
synchronized php-1.18/extensions/Translate/ 'Deploying translate [[rev:108451|r108451]]' |
[production] |
| 18:54 |
<nikerabbit> |
synchronized php-1.18/languages/messages/MessagesEn.php 'Updating messagesEn' |
[production] |
| 18:42 |
<nikerabbit> |
synchronized php-1.18/extensions/ParserFunctions/ParserFunctions.i18n.magic.php 'Deploying [[rev:108449|r108449]]' |
[production] |
| 18:34 |
<nikerabbit> |
synchronized php-1.18/extensions/WikimediaMessages/WikimediaGrammarForms.php 'Deploying [[rev:108433|r108433]]' |
[production] |
| 18:30 |
<nikerabbit> |
synchronized p/extensions/WebFonts/ 'Updating WebFonts [[rev:108447|r108447]]' |
[production] |
| 18:21 |
<nikerabbit> |
synchronized php-1.18/extensions/Narayam/ 'Syncing Narayam' |
[production] |
| 18:18 |
<Nikerabbit> |
running Narayam preference migration script |
[production] |
| 17:30 |
<jeremyb> |
the time is now 17:30:30 UTC |
[production] |
| 17:20 |
<RoanKattouw> |
Installing (!) NTP on wikitech |
[production] |
| 17:16 |
<jeremyb> |
the time is now 17:19:30 UTC |
[production] |
| 02:01 |
<LocalisationUpdate> |
completed (1.18) at Mon Jan 9 02:04:47 UTC 2012 |
[production] |
|
2012-01-08
§
|
| 23:20 |
<Reedy> |
For some reason cp1001-10042 weren't listed in CommonSettings.php XFF, but (at least) 1042 was in service, meaning edits were attributed to it |
[production] |
| 23:18 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Add cp1001-cp1041' |
[production] |
| 23:10 |
<reedy> |
synchronized wmf-config/CommonSettings.php 'Add cp1042 to XFF' |
[production] |
| 21:47 |
<rainman-sr> |
killed broken search indexer thread on searchidx1 (please note searchidx1 is no longer in use!), and restarted incremental indexing on searchidx2 which was somehow broken |
[production] |
| 21:43 |
<rainman-sr> |
someone started incremental updating on searchidx1 ??!! |
[production] |
| 14:54 |
<apergos> |
removed old puppet lockfile on brewster, ran by hand |
[production] |
| 14:47 |
<apergos> |
cleared out some very large squid logs on brewster, (basically all of them) plus lighty logs, disk was full. restarted squid manually |
[production] |
| 02:01 |
<LocalisationUpdate> |
completed (1.18) at Sun Jan 8 02:05:11 UTC 2012 |
[production] |
| 00:43 |
<tfinc> |
killing long running show_bug.cgi procs on kaulen |
[production] |
|
2012-01-06
§
|
| 23:22 |
<RobH> |
working rt1549 lvs1003 may flap, it is presently not in service due to possible hdd failure |
[production] |
| 22:55 |
<binasher> |
db22 is back in s4 |
[production] |
| 22:55 |
<asher> |
synchronized wmf-config/db.php 'adding db22 back to s4' |
[production] |
| 21:41 |
<RobH> |
db1029 powering back up with ssd testing hardware installed |
[production] |
| 21:35 |
<RobH> |
db1029 coming down for ssd testing |
[production] |
| 21:26 |
<RobH> |
cp1014 and cp1019 hdd controller cables replaced (removed for testing controllers), both can be used normally |
[production] |
| 21:19 |
<binasher> |
restoring db22 from a live hotbackup of db1038 |
[production] |
| 21:18 |
<RobH> |
es1002 back ready for service use per #2220: replace original RAID card in es1002 |
[production] |
| 21:05 |
<binasher> |
putting db51 into production as an s4 slave |
[production] |
| 21:05 |
<asher> |
synchronized wmf-config/db.php 'adding db51 as an s4 slave' |
[production] |
| 20:57 |
<binasher> |
started slaving db51 off of db31 |
[production] |
| 20:21 |
<RobH> |
rt2226 - redeploy db22 for asher |
[production] |
| 20:19 |
<RobH> |
db22 reinstalled and booting into OS. No puppet runs yet, now its Asher's problem ;] |
[production] |
| 20:04 |
<RobH> |
db22 reinstalling |
[production] |
| 19:24 |
<binasher> |
started innodb hot backup of db1038 to db51 |
[production] |
| 18:43 |
<maplebed> |
s4 database rotation complete. outage duration 36 minutes. |
[production] |
| 18:37 |
<maplebed> |
pushed out new db.php setting s4 to read-write |
[production] |
| 18:37 |
<ben> |
synchronized wmf-config/db.php |
[production] |
| 18:35 |
<maplebed> |
db31 made read-write as the new master for s4 |
[production] |
| 18:31 |
<maplebed> |
old master for s4 log file db22-bin.000106 log pos 631618956 |
[production] |
| 18:30 |
<maplebed> |
new master for s4: db31, log file db31-bin.000213 log pos is 205612709 |
[production] |
| 18:24 |
<asher> |
synchronized wmf-config/db.php 'setting s4 to read only, preparing to make db31 master' |
[production] |
| 18:22 |
<Reedy> |
Commons having db issues, db22 (s4 master) has a disk issue |
[production] |