|
2014-02-06
ยง
|
| 22:41 |
<springle> |
synchronized wmf-config/db-eqiad.php 's1 db1055 resume groupLoadsByDB' |
[production] |
| 22:28 |
<demon> |
synchronized php-1.23wmf13/extensions/Math 'Unfataling wmf13 Math too' |
[production] |
| 22:28 |
<demon> |
updated /a/common/php-1.23wmf13 to {{Gerrit|I5ba48ca51}}: Reverting Math to known-good 2b8534793fad9db18fcdb9621dc8d79ff36fdeb1 |
[production] |
| 22:16 |
<demon> |
synchronized php-1.23wmf12/extensions/Math 'New math causes cache stampedes but without fatals' |
[production] |
| 21:54 |
<demon> |
synchronized php-1.23wmf13/extensions/Math 'Revert to known-good 2b85347 from wmf11' |
[production] |
| 21:52 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedia back to 1.23wmf12, group0 to 1.23wmf13, Math reverted to 1.23wmf11 state in both branches |
[production] |
| 21:51 |
<demon> |
updated /a/common/php-1.23wmf13 to {{Gerrit|I233eaf25c}}: Revert "Add sequence support for externallinks table" |
[production] |
| 21:40 |
<demon> |
updated /a/common/php-1.23wmf12 to {{Gerrit|I91e982773}}: Update Echo and Flow |
[production] |
| 21:39 |
<demon> |
synchronized php-1.23wmf12/extensions/Math/ 'Revert to known-good 2b85347 from wmf11' |
[production] |
| 21:17 |
<springle> |
synchronized wmf-config/db-eqiad.php 's2 repool db1009 warm up' |
[production] |
| 19:46 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Revert earlier version changes |
[production] |
| 19:31 |
<reedy> |
started scap: run 2, should be a noop |
[production] |
| 19:28 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 wikis to 1.23wmf13 |
[production] |
| 19:13 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedias to 1.23wmf12 |
[production] |
| 18:36 |
<demon> |
synchronized wmf-config/CirrusSearch-labs.php 'Enabled interwiki searches for beta -- no-op in prod' |
[production] |
| 17:45 |
<paravoid> |
depooling mw1165 |
[production] |
| 17:43 |
<Reedy> |
mw1165 is segfaulting a lot |
[production] |
| 17:18 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki back to 1.23wmf12 till window |
[production] |
| 17:17 |
<reedy> |
finished scap: testwiki to 1.23wmf13 and build l10n cache (duration: 69m 51s) |
[production] |
| 16:50 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
| 16:48 |
<Nemo_bis> |
#Wikipedia and #Wikimedia wikis have been partly down for about 10 min, now recovered/ing |
[production] |
| 16:41 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
| 16:07 |
<reedy> |
started scap: testwiki to 1.23wmf13 and build l10n cache |
[production] |
| 16:04 |
<reedy> |
started scap: testwiki to 1.23wmf13 and build l10n cache |
[production] |
| 16:00 |
<reedy> |
started scap: testwiki to 1.23wmf13 and build l10n cache |
[production] |
| 15:57 |
<hashar> |
restarted jenkins by mistake :-( |
[production] |
| 15:28 |
<reedy> |
updated /a/common to {{Gerrit|I1e5ea52ae}}: s2 increase db1024 load after warm up |
[production] |
| 13:13 |
<paravoid> |
US/Canada pacific states being served by ulsfo |
[production] |
| 12:43 |
<paravoid> |
pointing the rest of East Asia (except CN) to ulsfo |
[production] |
| 11:14 |
<hashar> |
jenkins: added label <tt>hasTox</tt> on integration-slave01.pmtpa.wmflabs. Will let us run tox based Jenkins jobs there. |
[production] |
| 11:03 |
<springle> |
synchronized wmf-config/db-eqiad.php 's2 increase db1024 load after warm up' |
[production] |
| 07:29 |
<Ryan_Lane> |
redeploying parsoid/deploy on wtp* |
[production] |
| 07:01 |
<springle> |
xtrabackup clone db1018 to db1009 (take #2) |
[production] |
| 06:48 |
<springle> |
synchronized wmf-config/db-eqiad.php 's2 pool db1024 warm up' |
[production] |
| 05:48 |
<ori> |
varnish on cp1054: CPU wait spiked at 05:27. dmesg|tail: XFS: possible memory allocation deadlock in kmem_alloc. not investigating further. |
[production] |
| 05:45 |
<ori> |
finished scap: no new code. testing scap changes. (again.) (duration: 05m 00s) |
[production] |
| 05:40 |
<ori> |
started scap: no new code. testing scap changes. (again.) |
[production] |
| 05:15 |
<springle> |
restart labsdb1003 mariadb instances |
[production] |
| 05:09 |
<Ryan_Lane> |
scratch that, will redploy parsoid/deploy in about an hour |
[production] |
| 05:08 |
<Ryan_Lane> |
redeploying parsoid/deploy on wtp* |
[production] |
| 04:28 |
<ori> |
finished scap: no new code. testing scap changes. (duration: 04m 35s) |
[production] |
| 04:24 |
<ori> |
started scap: no new code. testing scap changes. |
[production] |
| 03:22 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-02-06 03:22:26+00:00 |
[production] |
| 03:10 |
<springle> |
xtrabackup clone db1018 to db1024 |
[production] |
| 02:45 |
<springle> |
xtrabackup clone db1034 to db1009 |
[production] |
| 02:44 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-02-06 02:44:10+00:00 |
[production] |
| 02:23 |
<springle> |
synchronized wmf-config/db-eqiad.php 's2 depool db1009 schema changes' |
[production] |
| 02:21 |
<LocalisationUpdate> |
completed (1.23wmf12) at 2014-02-06 02:21:47+00:00 |
[production] |
| 02:01 |
<springle> |
synchronized wmf-config/db-eqiad.php 's2 repool db1034 warm up' |
[production] |
| 01:06 |
<mutante> |
apt-get remove libnet-stomp-perl on neon, i just removed that from puppet but didn't think it should stay in as an "absent" package forever |
[production] |