|
2014-03-13
§
|
| 16:59 |
<apergos> |
restarted gerrit which had been slow |
[production] |
| 16:55 |
<robh> |
gerrit is just really slow, but not quite down. |
[production] |
| 16:49 |
<bd808> |
Started scap: testwiki to php-1.23wmf18 and rebuild l10n cache |
[production] |
| 16:21 |
<paravoid> |
deactivating Tele2 from eqiad, blackholing traffic |
[production] |
| 16:08 |
<paravoid> |
fixing pybal for mw1208/mw1209/mw1210 move; same issue as last time |
[production] |
| 15:45 |
<paravoid> |
announcing only Tampa's /23 from pmtpa/sdtpa |
[production] |
| 15:04 |
<^d> |
restarting gerrit process, ytterbium was swapping |
[production] |
| 12:40 |
<cmjohnson1> |
shutting down and relocating mw1208, mw1209 and mw1210 to row D |
[production] |
| 08:49 |
<paravoid> |
API slowness (affecting Parsoid/VE, among others) there since ~12:00 UTC found and fixed |
[production] |
| 07:42 |
<RoanKattouw> |
Reverted API pybal weights back to original values; apparently makes sense given amount of memory |
[production] |
| 07:05 |
<RoanKattouw> |
Changed pybal weights for eqiad API cluster to # of CPUs on each machine; weights were backwards (machines with fewer CPUs had higher weights) |
[production] |
| 03:23 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Mar 13 03:23:13 UTC 2014 (duration 23m 12s) |
[production] |
| 02:34 |
<LocalisationUpdate> |
completed (1.23wmf17) at 2014-03-13 02:33:54+00:00 |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.23wmf16) at 2014-03-13 02:17:46+00:00 |
[production] |
| 00:14 |
<awight> |
updated crm from 445f1f0ce489e65bf3765a23403f5b52b77f23cf to 2353dbcda410645e02dd9f049ff9e1eb341d2f16 |
[production] |
|
2014-03-12
§
|
| 23:50 |
<mwalker> |
synchronized php-1.23wmf17/extensions/CentralNotice/ |
[production] |
| 23:49 |
<mwalker> |
synchronized php-1.23wmf16/extensions/CentralNotice/ |
[production] |
| 23:21 |
<mwalker> |
synchronized php-1.23wmf17/extensions/CentralNotice |
[production] |
| 23:20 |
<mwalker> |
synchronized php-1.23wmf16/extensions/CentralNotice |
[production] |
| 21:57 |
<bd808> |
Finished scap: another no-diff scap to test script changes (duration: 03m 25s) |
[production] |
| 21:54 |
<bd808> |
Started scap: another no-diff scap to test script changes |
[production] |
| 21:54 |
<ottomata> |
initiated controlled shutdown of an21, promoting an22 to leader of all partitions |
[production] |
| 21:51 |
<bd808> |
Finished scap: no-diff scap to test script changes (duration: 21m 42s) |
[production] |
| 21:30 |
<bd808> |
Started scap: no-diff scap to test script changes |
[production] |
| 21:14 |
<mutante> |
welcome to root, Chase, added key to root-auth-keys |
[production] |
| 20:05 |
<Coren> |
restarting parsoids as requested by gwicke |
[production] |
| 20:02 |
<gwicke> |
deployed Parsoid 004c7acc with deploy f97820a2; restart todo |
[production] |
| 19:12 |
<yurik> |
synchronized php-1.23wmf16/extensions/ZeroRatedMobileAccess/ |
[production] |
| 19:11 |
<yurik> |
synchronized php-1.23wmf17/extensions/ZeroRatedMobileAccess/ |
[production] |
| 19:06 |
<mutante> |
graceful apache on gallium |
[production] |
| 17:36 |
<yurik> |
synchronized php-1.23wmf17/extensions/ZeroRatedMobileAccess/ |
[production] |
| 17:27 |
<yurik> |
synchronized php-1.23wmf16/extensions/ZeroRatedMobileAccess/ |
[production] |
| 17:17 |
<awight> |
updated crm from f0837090e918b7016c07cb366b33c8d8f0d1c661 to 445f1f0ce489e65bf3765a23403f5b52b77f23cf |
[production] |
| 17:02 |
<reedy> |
synchronized wmf-config/ 'noop' |
[production] |
| 16:56 |
<reedy> |
synchronized docroot and w |
[production] |
| 16:56 |
<reedy> |
synchronized wmf-config/ 'I6f1f3f8af5b97aa0e537fbae308ce27b28071894' |
[production] |
| 16:10 |
<ottomata> |
initiating controlled shutdown of analytics1021 kafka broker to do some load testing and also fix runtime java version |
[production] |
| 15:56 |
<cmjohnson1> |
ms-be1005 going down to fix mgmt |
[production] |
| 15:06 |
<hoo> |
syncing to mw120[1-3] failed |
[production] |
| 14:58 |
<hoo> |
synchronized php-1.23wmf17/extensions/CentralAuth/ 'Fix global account deletion' |
[production] |
| 14:38 |
<springle> |
synchronized wmf-config/db-eqiad.php 's2 db1063 full steam' |
[production] |
| 14:24 |
<paravoid> |
deploying new swift ring @ eqiad, setting weight from 100 to 2000 on all disks |
[production] |
| 13:01 |
<cmjohnson1> |
shutting down and relocating mw1201, mw1202, mw1203 to d5-eqiad |
[production] |
| 10:41 |
<maxsem> |
synchronized wmf-config/mobile.php 'https://gerrit.wikimedia.org/r/118227' |
[production] |
| 10:17 |
<springle> |
started s4 dump for toolserver on db72 /a |
[production] |
| 09:52 |
<springle> |
synchronized wmf-config/db-eqiad.php 's2 pool db1063, warm up' |
[production] |
| 09:45 |
<hashar> |
rerestarting Jenkins |
[production] |
| 09:16 |
<hashar> |
kill -9 of Jenkins since it is unresponsive |
[production] |
| 09:13 |
<hashar> |
restarting Jenkins |
[production] |
| 09:12 |
<hashar> |
Jenkins broken again! Good morning. |
[production] |