|
2012-11-22
§
|
| 12:06 |
<hashar> |
restarted Jenkins to load in the new zuul jobs. |
[production] |
| 09:23 |
<apergos> |
fixed perms on srv284 php-1.*wmf* and wmf-config, test (were 700 instead of 777) |
[production] |
| 02:24 |
<logmsgbot_> |
LocalisationUpdate completed (1.21wmf4) at Thu Nov 22 02:24:22 UTC 2012 |
[production] |
| 01:29 |
<mutante> |
sync-apache srv284-only, start apache (was missing all.conf), repool |
[production] |
| 01:15 |
<mutante> |
rebooting srv193 (test.wp) for upgrade |
[production] |
| 00:56 |
<mutante> |
repooled mw58,mw59 (upgrades) srv284 (hw ticket was resolved, reinstalled) |
[production] |
| 00:07 |
<mutante> |
powercycling kaulen, this time no console output at all |
[production] |
|
2012-11-21
§
|
| 23:47 |
<mutante> |
tmp. depooling and rebooting a few mw5x servers for kernel upgrades, one by one |
[production] |
| 23:37 |
<logmsgbot_> |
preilly synchronized php-1.21wmf4/extensions/ZeroRatedMobileAccess 'update post deploy' |
[production] |
| 23:34 |
<logmsgbot_> |
preilly synchronized php-1.21wmf3/extensions/ZeroRatedMobileAccess 'update post deploy' |
[production] |
| 22:58 |
<mutante> |
disabled swap on kaulen per Tim's advice |
[production] |
| 22:55 |
<mutante> |
bugzilla back up |
[production] |
| 22:50 |
<mutante> |
powercycling kaulen |
[production] |
| 22:25 |
<logmsgbot_> |
asher synchronized wmf-config/throttle.php 'increased api limit for [[bugzilla:42319|bug 42319]]' |
[production] |
| 21:51 |
<mutante> |
installing package upgrades in the mw55-mw99 range |
[production] |
| 21:17 |
<logmsgbot_> |
preilly Finished syncing Wikimedia installation... : update zero rated mobile access |
[production] |
| 20:56 |
<Ryan_Lane> |
deploying Change If5f6bc33: ([[bugzilla:42334|bug 42334]]) on labsconsole |
[production] |
| 20:47 |
<Ryan_Lane> |
applying changes to fix no creds issue when setting preferences on labsconsole |
[production] |
| 20:47 |
<logmsgbot_> |
preilly Started syncing Wikimedia installation... : update zero rated mobile access |
[production] |
| 20:34 |
<logmsgbot_> |
preilly synchronized php-1.21wmf4/extensions/ZeroRatedMobileAccess 'update post deploy' |
[production] |
| 20:34 |
<logmsgbot_> |
preilly synchronized php-1.21wmf3/extensions/ZeroRatedMobileAccess 'update post deploy' |
[production] |
| 20:00 |
<logmsgbot_> |
reedy rebuilt wikiversions.cdb and synchronized wikiversions files: rest of wikipedias to 1.21wmf4 |
[production] |
| 19:24 |
<mutante> |
installing package upgrades in the srv193-srv199 range |
[production] |
| 18:55 |
<mutante> |
re-signing srv284 on puppetmaster, package installs... |
[production] |
| 18:45 |
<AaronSchulz> |
Copying over all math/timeline files to nas1 that are not already there for DR |
[production] |
| 18:34 |
<AaronSchulz> |
prepared all wikivoyage file directories and containers |
[production] |
| 17:41 |
<Reedy> |
Running scap-recompile on all hosts in mediawiki-installation group |
[production] |
| 16:38 |
<mutante> |
powering on analytics1007 |
[production] |
| 16:29 |
<mutante> |
reinstalling srv284 "last lucid standing" |
[production] |
| 16:24 |
<mark> |
Takeback by nas1001-a completed, snapmirror relationships back in sync |
[production] |
| 16:15 |
<mark> |
Aborted synchronous snapmirror relationships between nas1-a and nas1001-a to initiate cf giveback |
[production] |
| 16:08 |
<mutante> |
powercycling srv284 |
[production] |
| 16:05 |
<mutante> |
remove known troublemakers srv266 and srv284 from dsh group "apaches" |
[production] |
| 15:50 |
<mark> |
Initiated takeover of nas1001-a to nas1001-b |
[production] |
| 15:49 |
<mark> |
nas1001-b back up and running |
[production] |
| 14:46 |
<mark> |
Halted nas1001-b for NIC upgrade |
[production] |
| 09:18 |
<notpeter> |
repooling mw20-mw34 |
[production] |
| 07:50 |
<notpeter> |
repooling mw70-mw74 |
[production] |
| 07:21 |
<apergos> |
restarted lucene search on search14 |
[production] |
| 03:01 |
<notpeter> |
depooling mw70-mw74 for upgrade to precise |
[production] |
| 03:01 |
<notpeter> |
depooling mw20-mw34 for upgrade to precise |
[production] |
| 02:45 |
<logmsgbot_> |
LocalisationUpdate completed (1.21wmf3) at Wed Nov 21 02:45:42 UTC 2012 |
[production] |
| 02:24 |
<logmsgbot_> |
LocalisationUpdate completed (1.21wmf4) at Wed Nov 21 02:24:47 UTC 2012 |
[production] |
| 02:17 |
<K4-713> |
updated payments cluster to 6ea97a5cbc06 |
[production] |
| 00:40 |
<logmsgbot_> |
preilly synchronized php-1.21wmf4/extensions/ZeroRatedMobileAccess 'update post deploy' |
[production] |
| 00:37 |
<logmsgbot_> |
preilly synchronized php-1.21wmf3/extensions/ZeroRatedMobileAccess 'update post deploy' |
[production] |
| 00:06 |
<logmsgbot_> |
maxsem synchronized php-1.21wmf4/extensions/MobileFrontend 'https://gerrit.wikimedia.org/r/#/c/34440/' |
[production] |
| 00:03 |
<logmsgbot_> |
maxsem synchronized php-1.21wmf3/extensions/MobileFrontend 'https://gerrit.wikimedia.org/r/#/c/34440/' |
[production] |
| 00:03 |
<notpeter> |
depooling mw66-mw99 for upgrade to apache |
[production] |
| 00:03 |
<notpeter> |
depooling mw53-mw49 for upgrade to apache |
[production] |