|
2011-11-14
§
|
| 14:04 |
<RoanKattouw> |
Strike that, it is responding to SSH because neither the connection nor the setup time out. It's just terribly, terribly slow |
[production] |
| 14:02 |
<RoanKattouw> |
searchidx2 is dead, not responding to SSH. Ganglia shows elevated load |
[production] |
| 13:57 |
<mark> |
Set all esams text amssq (SSD) squids to CARP weight 30 |
[production] |
| 13:46 |
<mark> |
Depooled knsq1 and knsq2 from esams bits duty, preparing for decommissioning |
[production] |
| 13:26 |
<mark> |
Ran dist-upgrade and reboot on cp3001 and cp3002 |
[production] |
| 13:26 |
<mark> |
Setup interface aggregates on cp3001 and cp3002 |
[production] |
| 12:04 |
<mark> |
Deploying bits.eqiad |
[production] |
| 11:58 |
<mark> |
bits-lb.esams record addition wasn't picked up by pdns on reload during the previous update, quickly moving bits-lb back to bits.esams |
[production] |
| 11:53 |
<mark> |
Added bits-lb.esams.wikimedia.org DNS record, and updated geodns to use that instead of bits.esams |
[production] |
| 11:52 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Bug 31459 - Gender in user namespaces for Portuguese Wikipedia' |
[production] |
| 11:39 |
<mark> |
synchronized wmf-config/CommonSettings.php 'remove old Squids' |
[production] |
| 11:30 |
<mark> |
Renamed ganglia clusters/dirs on streber |
[production] |
| 11:22 |
<mark> |
Renamed ganglia RRD dirs on spence |
[production] |
| 10:35 |
<catrope> |
synchronized php-1.18/extensions/CentralNotice/special/SpecialBannerController.php '[[rev:102966|r102966]]' |
[production] |
| 10:34 |
<catrope> |
synchronized php-1.18/extensions/Vector/modules/ext.vector.collapsibleTabs.js '[[rev:102966|r102966]]' |
[production] |
| 09:58 |
<mark> |
Fixing ganglia's confusedness about $site |
[production] |
| 08:39 |
<catrope> |
synchronized php-1.18/extensions/Collection/Collection.body.php '[[rev:102950|r102950]]' |
[production] |
| 08:26 |
<RoanKattouw> |
I meant [[rev:102944|r102944]] |
[production] |
| 07:52 |
<catrope> |
synchronized php-1.18/extensions/Gadgets/Gadgets_body.php '[[rev:102994|r102994]]' |
[production] |
| 05:09 |
<awjr> |
synchroinzing payments cluster to [[rev:102939|r102939]] |
[production] |
| 03:31 |
<awjr> |
synchronizing payments cluster to [[rev:102935|r102935]] |
[production] |
| 02:24 |
<LocalisationUpdate> |
completed (1.18) at Mon Nov 14 02:27:03 UTC 2011 |
[production] |
| 01:57 |
<LocalisationUpdate> |
failed |
[production] |
|
2011-11-12
§
|
| 15:56 |
<apergos> |
removed some old snapshot files from ms7 (not actual snaps, but saved copies of them no longer useful), freeing up about 7T of space |
[production] |
| 15:47 |
<catrope> |
synchronized wmf-config/CommonSettings.php 'Re-enable TitleBlacklist on kwwiktionary' |
[production] |
| 15:45 |
<catrope> |
synchronized wmf-config/CommonSettings.php 'Temp disable TitleBlacklist on kwwiktionary' |
[production] |
| 09:44 |
<catrope> |
synchronized wmf-config/CommonSettings.php 'Replace Botswana account creation throttle exemption with one for Serbia' |
[production] |
| 06:04 |
<neilk> |
synchronized wmf-config/CommonSettings.php 'Goes w/ previous change -- re-enable Special:UploadWizard where it is in use (i.e. Commons)' |
[production] |
| 05:59 |
<neilk> |
synchronized wmf-config/CommonSettings.php 'Goes w/ previous change -- also need to disable multi-file select when disabling chunks' |
[production] |
| 05:51 |
<neilk> |
synchronized wmf-config/CommonSettings.php 'Disabling chunked uploads temporarily, according to Erik we seem to have not deployed the API that can handle them' |
[production] |
| 02:38 |
<awjr> |
synchronizing payments cluster to [[rev:102843|r102843]] |
[production] |
| 02:28 |
<LocalisationUpdate> |
completed (1.18) at Sat Nov 12 02:31:22 UTC 2011 |
[production] |
| 02:23 |
<demon> |
synchronized wmf-config/CommonSettings.php 'Set fallbackToAltUploadForm to true per Erik, tempoary site issues' |
[production] |
| 01:57 |
<LocalisationUpdate> |
failed |
[production] |
| 01:20 |
<aaron> |
synchronized wmf-config/flaggedrevs.php 'arwiki: disabled autopromote, added Portal and Annex namespaces to $wgFlaggedRevsNamespaces' |
[production] |
|
2011-11-11
§
|
| 19:06 |
<awjr> |
Syncronizing payments [[rev:102807|r102807]] |
[production] |
| 12:40 |
<catrope> |
synchronized php-1.18/languages/messages/MessagesEn.php '[[rev:102777|r102777]]' |
[production] |
| 10:22 |
<mark> |
Fixing up amssq57 |
[production] |
| 10:09 |
<apergos> |
traffic back to normal, let's see |
[production] |
| 10:06 |
<mark> |
csw1-esams reloaded, back up |
[production] |
| 09:51 |
<apergos> |
moved traffic back to tampa |
[production] |
| 09:43 |
<apergos> |
restored traffic so more tests can be done |
[production] |
| 08:56 |
<apergos> |
moved traffic to pmtpa, csw1 reboots reported from observium |
[production] |
| 03:43 |
<Ryan_Lane> |
finished gluster switchover for labs |
[production] |
| 03:32 |
<Ryan_Lane> |
rebooting virt2 to test gluster switchover |
[production] |