|
2014-02-06
§
|
| 05:48 |
<ori> |
varnish on cp1054: CPU wait spiked at 05:27. dmesg|tail: XFS: possible memory allocation deadlock in kmem_alloc. not investigating further. |
[production] |
| 05:45 |
<ori> |
finished scap: no new code. testing scap changes. (again.) (duration: 05m 00s) |
[production] |
| 05:40 |
<ori> |
started scap: no new code. testing scap changes. (again.) |
[production] |
| 05:15 |
<springle> |
restart labsdb1003 mariadb instances |
[production] |
| 05:09 |
<Ryan_Lane> |
scratch that, will redploy parsoid/deploy in about an hour |
[production] |
| 05:08 |
<Ryan_Lane> |
redeploying parsoid/deploy on wtp* |
[production] |
| 04:28 |
<ori> |
finished scap: no new code. testing scap changes. (duration: 04m 35s) |
[production] |
| 04:24 |
<ori> |
started scap: no new code. testing scap changes. |
[production] |
| 03:22 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-02-06 03:22:26+00:00 |
[production] |
| 03:10 |
<springle> |
xtrabackup clone db1018 to db1024 |
[production] |
| 02:45 |
<springle> |
xtrabackup clone db1034 to db1009 |
[production] |
| 02:44 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-02-06 02:44:10+00:00 |
[production] |
| 02:23 |
<springle> |
synchronized wmf-config/db-eqiad.php 's2 depool db1009 schema changes' |
[production] |
| 02:21 |
<LocalisationUpdate> |
completed (1.23wmf12) at 2014-02-06 02:21:47+00:00 |
[production] |
| 02:01 |
<springle> |
synchronized wmf-config/db-eqiad.php 's2 repool db1034 warm up' |
[production] |
| 01:06 |
<mutante> |
apt-get remove libnet-stomp-perl on neon, i just removed that from puppet but didn't think it should stay in as an "absent" package forever |
[production] |
| 01:01 |
<ori> |
finished scap: no-diff scap to test script changes (duration: 02m 34s) |
[production] |
| 00:58 |
<ori> |
started scap: no-diff scap to test script changes |
[production] |
| 00:44 |
<ebernhardson> |
finished deploying Special:Notifications fix to Echo and Flow |
[production] |
| 00:44 |
<ebernhardson> |
synchronized php-1.23wmf11/extensions/Flow/ 'Update flow for Special:Notifications fix' |
[production] |
| 00:43 |
<ebernhardson> |
synchronized php-1.23wmf11/extensions/Echo/ 'Update echo for Special:Notifications fix' |
[production] |
| 00:38 |
<ebernhardson> |
synchronized php-1.23wmf12/extensions/Flow/ 'Update flow for Special:Notifications fix' |
[production] |
| 00:37 |
<ebernhardson> |
synchronized php-1.23wmf12/extensions/Echo/ 'Update echo for Special:Notifications fix' |
[production] |
| 00:04 |
<ori> |
finished scap: (no message) (duration: 29m 26s) |
[production] |
| 00:04 |
<ori> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
|
2014-02-05
§
|
| 23:34 |
<ori> |
started scap: (no message) |
[production] |
| 22:11 |
<ottomata> |
stopping puppet on analytics1021. Trying to get it to catch up on replica lag |
[production] |
| 20:13 |
<mwalker> |
updated fundraising civicrm from ec1e1fc75193ac8c0e6237fc0b80f58b3f8d55fc to 309a69545b4c89b323bffc500f71bdee82c35e42 for logging updates |
[production] |
| 18:57 |
<yurik> |
synchronized php-1.23wmf12/extensions/ZeroRatedMobileAccess/ |
[production] |
| 18:49 |
<yurik> |
synchronized php-1.23wmf11/extensions/ZeroRatedMobileAccess/ |
[production] |
| 18:38 |
<yurik> |
synchronized docroot/bits/WikipediaMobileFirefoxOS |
[production] |
| 14:11 |
<paravoid> |
ms-be1002/sdd: megacli -DiscardPreservedCache, -CfgEachDskRaid0, puppet run |
[production] |
| 13:57 |
<paravoid> |
remove XO-Level3 avoided as-path from cr1/2-eqiad despite no ticket reply; seems to work now |
[production] |
| 13:57 |
<hashar> |
compressing Jenkins console logs on gallium.wikimedia.org using gzip -9 |
[production] |
| 08:47 |
<ori> |
finished scap: (no message) (duration: 03m 31s) |
[production] |
| 08:47 |
<ori> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
| 08:44 |
<ori> |
started scap: (no message) |
[production] |
| 08:17 |
<Ryan_Lane> |
restart parsoid on wtp1012 |
[production] |
| 08:14 |
<Ryan_Lane> |
stopping parsoid on wtp1012 shortly |
[production] |
| 07:53 |
<ori> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
| 07:51 |
<ori> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
| 07:15 |
<ori> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
| 06:59 |
<ori> |
finished scap: (no message) (duration: 00m 35s) |
[production] |
| 06:59 |
<ori> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
| 06:59 |
<ori> |
started scap: (no message) |
[production] |
| 05:52 |
<springle> |
synchronized wmf-config/db-pmtpa.php 'update pmtpa config for s1 master change' |
[production] |
| 05:47 |
<springle> |
synchronized wmf-config/db-eqiad.php 'switch s1 master to db1052' |
[production] |
| 05:32 |
<springle> |
synchronized wmf-config/db-eqiad.php 'prepare for s1 master rotation db1056 to db1052' |
[production] |
| 05:18 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-02-05 05:18:46+00:00 |
[production] |
| 05:07 |
<ori> |
Last sync-file: connect to host mw1163 port 22: Connection timed out |
[production] |