|
2014-01-22
§
|
| 23:41 |
<hasharMeeting> |
Going to upgrade Zuul, need a patch to make it send Cache-Control: no-cache so we can migrate Zuul service behind misc varnish (being done with RobH) |
[production] |
| 22:58 |
<ottomata> |
restarted icinga (a while ago) and puppet freshness checks are all boogery(?) will check up on this tomorrow |
[production] |
| 22:51 |
<RobH> |
doc.wikimedia.org updates complete, once dns re-propagates it should be again reachable over https |
[production] |
| 22:43 |
<RobH> |
doc.wikimedia.org and doc.mediawiki.org relocating behind misc-web-lb. dns ttl was set to 5 minutes yesterday, so this change should result in less than 5 minutes of unreachability by anyone |
[production] |
| 22:09 |
<reedy> |
updated /a/common to {{Gerrit|Ie95fdf8e9}}: Adjust linkpurge and renderfile limits |
[production] |
| 21:26 |
<matanya> |
git.wikimedia.org is back up |
[production] |
| 20:44 |
<RobH> |
all plugins updated except caching cuz i hate it and i plan to move it behind misc-web-lb |
[production] |
| 20:42 |
<RobH> |
blog updated |
[production] |
| 19:40 |
<matanya> |
git.wikimedia.org is down |
[production] |
| 19:32 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 17:40 |
<RobH> |
virt0 puppet agent re-enabled and resumed normal service |
[production] |
| 17:05 |
<RobH> |
issues with change, don't re-enable puppet agent on virt0 yet |
[production] |
| 16:59 |
<RobH> |
rolling certificate changes to virt1000 and virt0. virt0 puppet agent disabled for moment |
[production] |
| 16:33 |
<Nemo_bis> |
still no data in Global_JobQueue_length ganglia graph on hume (3 days now) |
[production] |
| 14:34 |
<csteipp> |
synchronized php-1.23wmf10/includes/media 'bug60339' |
[production] |
| 14:33 |
<csteipp> |
synchronized php-1.23wmf11/includes/media 'bug60339' |
[production] |
| 11:18 |
<apergos> |
powercycled kaulen, it had died the swap death |
[production] |
| 10:02 |
<apergos> |
restarted mw1206 a little while ago, it was inaccessible even via mgmt |
[production] |
| 06:31 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 05:37 |
<ori> |
restarting hafnium for kernel upgrade |
[production] |
| 03:33 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-01-22 03:33:21+00:00 |
[production] |
| 00:55 |
<marktraceur> |
deployed config for jsduck jobs for MultimediaViewer |
[production] |
| 00:22 |
<demon> |
synchronized php-1.23wmf11/extensions/CirrusSearch 'Cirrus to master, with better job queues' |
[production] |
| 00:21 |
<demon> |
synchronized php-1.23wmf10/extensions/CirrusSearch 'Cirrus to master, with better job queues' |
[production] |
| 00:18 |
<demon> |
synchronized php-1.23wmf10/extensions/Elastica 'Elastica to master' |
[production] |
| 00:15 |
<demon> |
synchronized php-1.23wmf10/extensions/CirrusSearch 'Rolling back Cirrus -- fatals' |
[production] |
| 00:11 |
<demon> |
synchronized php-1.23wmf10/extensions/CirrusSearch 'Cirrus to master, with better job queues' |
[production] |
|
2014-01-21
§
|
| 23:55 |
<marktraceur> |
deployed documentation jenkins jobs for MultimediaViewer |
[production] |
| 23:52 |
<awight> |
crm updated from e82d2c5e64e67bee9bfa1f703fc94602addea2cb to f71526fa021fe3963a5d94151881f5a8a5c40dab |
[production] |
| 20:24 |
<cmjohnson1> |
removing iptable rules for loudon and erzurumi |
[production] |
| 20:20 |
<awight> |
updated listeners from dc21b3bf6560402238f9093f1dfbafe22be0a511 to 5e18193c5bb703e44c5cda22009ca97c295973e3 |
[production] |
| 20:06 |
<awight> |
updated listeners from 888561ae0ee5a1d5f86da98ab3062f674726717f to dc21b3bf6560402238f9093f1dfbafe22be0a511 |
[production] |
| 19:51 |
<awight> |
updated listeners from 7c5396c5954e28fe58033008a3b34ebf076a39a5 to 888561ae0ee5a1d5f86da98ab3062f674726717f |
[production] |
| 19:42 |
<ori> |
synchronized wmf-config/CommonSettings.php 'Ie9ff56755: Add wmgUniversalLanguageSelectorDefault (2/2)' |
[production] |
| 19:42 |
<ori> |
synchronized wmf-config/InitialiseSettings.php 'Ie9ff56755: Add wmgUniversalLanguageSelectorDefault' |
[production] |
| 19:41 |
<ori> |
updated /a/common to {{Gerrit|Ie9ff56755}}: Add 'wmgUniversalLanguageSelectorDefault' |
[production] |
| 19:17 |
<ori> |
synchronized php-1.23wmf10/extensions/UniversalLanguageSelector/Resources.php 'I05c76e478: Make ext.uls.mediawiki depend upon ext.uls.init' |
[production] |
| 19:08 |
<ori> |
sync-file: mw1206: ssh: connect to host mw1206 port 22: No route to host |
[production] |
| 19:07 |
<ori> |
synchronized php-1.23wmf11/extensions/UniversalLanguageSelector/Resources.php 'I05c76e478: Make ext.uls.mediawiki depend upon ext.uls.init' |
[production] |
| 18:58 |
<awight> |
updated SmashPig to 43a27bd379632792bd7fc3ffc26c859d6dbbe0d6 |
[production] |
| 17:06 |
<demon> |
finished scap: No code updates, just rebuilding all i18n (duration: 58m 29s) |
[production] |
| 16:13 |
<demon> |
started scap: No code updates, just rebuilding all i18n |
[production] |
| 11:59 |
<twkozlowski> |
bits application servers back to normal since around 11:46 UTC |
[production] |
| 11:32 |
<ori> |
bits application server overload due to I71b70d8ee |
[production] |