|
2013-05-31
§
|
| 21:33 |
<hashar> |
Zuul / Jenkins had some delay most probably related to a lot of patchsets being sent at the same time. Zuul catching up right now. |
[production] |
| 21:15 |
<binasher> |
completed echo event_page_id schema migrations |
[production] |
| 20:49 |
<binasher> |
restarted gerrit |
[production] |
| 19:53 |
<AaronSchulz> |
Running a terbium script to copy missing files to swift as well as ceph |
[production] |
| 19:24 |
<aaron> |
synchronized php-1.22wmf5/maintenance/copyFileBackend.php '82352634594d0a67d3b9f6597c3bb2c0911f8028' |
[production] |
| 19:22 |
<aaron> |
synchronized php-1.22wmf4/maintenance/copyFileBackend.php '3d0e3f8e4d09d4a2462f583c25dd284e6ee3e466' |
[production] |
| 14:41 |
<reedy> |
synchronized php-1.22wmf4/extensions/SecurePoll/ |
[production] |
| 14:40 |
<reedy> |
synchronized php-1.22wmf5/extensions/SecurePoll/ |
[production] |
| 14:33 |
<Reedy> |
Created bv2013_edit tables on all wikis |
[production] |
| 14:29 |
<reedy> |
synchronized php-1.22wmf4/extensions/SecurePoll/ |
[production] |
| 14:28 |
<reedy> |
synchronized php-1.22wmf5/extensions/SecurePoll/ |
[production] |
| 12:04 |
<apergos> |
uh... it would help if I looked on tin, the actual deploy host so nm. anyways texvc now working |
[production] |
| 12:00 |
<apergos> |
ran scap-recompile to get texvc built for 1.22wmf5 and btw this had to be done from fenari, there's no dsh on terbium |
[production] |
| 06:56 |
<apergos> |
ceph osds 50 and 132 on ms-fe1002 are logging 'slow requests', don't know how to restart specific osds in bobtail though |
[production] |
| 06:42 |
<Aaron|home> |
Running a script on terbium to list out files that were updated (overwritten) but are not synced on all wikis (it found none on commons/enwiki) |
[production] |
| 05:25 |
<Tim> |
on msfe1001-1004: disabled apache_status ganglia module |
[production] |
| 04:17 |
<Tim> |
on nickel: restarting gmetad to see if that fixes ceph cluster reporting |
[production] |
| 04:01 |
<aaron> |
synchronized wmf-config/filebackend.php 'Disabled ceph backends and switched to just swift.' |
[production] |
| 02:31 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Fri May 31 02:31:40 UTC 2013 |
[production] |
| 02:11 |
<LocalisationUpdate> |
completed (1.22wmf4) at Fri May 31 02:11:03 UTC 2013 |
[production] |
| 02:06 |
<LocalisationUpdate> |
completed (1.22wmf5) at Fri May 31 02:06:06 UTC 2013 |
[production] |
| 00:10 |
<mwalker> |
Updating payments deploy from dce20cbeb501a9945d97b2e64c59dbb4dd0a9620 to cf2d1639a9a8b2854d5302db2aba86199b4e4101 for RTBT attempted fix |
[production] |
| 00:06 |
<kaldari> |
Finished syncing Wikimedia installation... : |
[production] |
|
2013-05-30
§
|
| 23:58 |
<LeslieCarr> |
fixing packages/security upgrades on db45 |
[production] |
| 23:56 |
<kaldari> |
Started syncing Wikimedia installation... : |
[production] |
| 23:54 |
<LeslieCarr> |
mw1171 appears to have a bad hard drive |
[production] |
| 22:07 |
<mflaschen> |
Finished syncing Wikimedia installation... : Deployment for E3 and E2 |
[production] |
| 21:54 |
<mflaschen> |
Started syncing Wikimedia installation... : Deployment for E3 and E2 |
[production] |
| 21:23 |
<LeslieCarr> |
powercycling mw16 |
[production] |
| 20:34 |
<cmjohnson1> |
ms-be106 replacing disk slot 6 |
[production] |
| 20:10 |
<cmjohnson1> |
es1001 replacing bad disk (slot 11) |
[production] |
| 20:05 |
<kaldari> |
Finished syncing Wikimedia installation... : |
[production] |
| 19:54 |
<LeslieCarr> |
enabling routing-engine on asw-c1-eqiad |
[production] |
| 19:53 |
<LeslieCarr> |
asw-c1-eqiad successfully replaced |
[production] |
| 19:47 |
<kaldari> |
Started syncing Wikimedia installation... : |
[production] |
| 19:21 |
<LeslieCarr> |
powering on new asw-c-eqiad switch -- possible disruption to row c connectivity (low risk) |
[production] |
| 19:19 |
<LeslieCarr> |
replacing asw-c1-eqiad with new switch |
[production] |
| 18:14 |
<bblack> |
updated vhtcpd deb to 0.0.6-1 on brewster, puppet will upgrade the varnishes |
[production] |
| 16:18 |
<andrewbogott_afk> |
updated OpenStackManager on virt0 to git head. |
[production] |
| 14:58 |
<paravoid> |
restarting ms-fe4 proxy |
[production] |
| 14:41 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'Enable SecurePoll on testwiki' |
[production] |
| 13:20 |
<RoanKattouw> |
(it's not broken, never mind, I'm just incapable of reading things) |
[production] |