|
2014-08-08
§
|
| 21:24 |
<Reedy> |
mw1130 seems to be dead (unresponsive to ping) |
[production] |
| 21:21 |
<reedy> |
Synchronized wmf-config/interwiki.cdb: (no message) (duration: 01m 04s) |
[production] |
| 21:06 |
<awight> |
deployed crm default/settings.php |
[production] |
| 17:07 |
<mutante> |
jenkins/puppet-compiler - granting new LDAP group "nda" the same rights already given to matanya (and wmde even has more) |
[production] |
| 16:06 |
<bblack> |
datacenter traffic mapping back to normal, varnish fix/wipe/restart/etc work on pause for the weekend in a stable state |
[production] |
| 16:02 |
<andrewbogott> |
merging https://gerrit.wikimedia.org/r/#/c/150273/ which affects every puppet log everywhere... |
[production] |
| 14:22 |
<mutante> |
RT - reverted permission change for access requests requestors per robh |
[production] |
| 13:50 |
<mutante> |
RT - granted permission to show ticket summary for role requestor in queue access-requests |
[production] |
| 12:49 |
<akosiaris> |
uploaded ruby-jsduck 5.3.4-1wmftrusty1 and ruby-rkelly-remix 0.0.6-1trusty1 on apt.wikimedia.org |
[production] |
| 12:33 |
<ori> |
testwiki up, judgement poor |
[production] |
| 12:28 |
<hashar> |
Jenkins: somehow the ArtifactDeployer plugin got upgraded on Aug 7th 20:57 UTC despite it being broken {{bug|69197}}. Attempting manual downgrade |
[production] |
| 12:13 |
<hashar> |
reloading Jenkins |
[production] |
| 12:07 |
<akosiaris> |
ifconfig br0 0.0.0.0 on platinum to get rid of the IP on that interface and have facter work more reliably. This does not matter right now as it is an evaluation machine but logging it for completeness |
[production] |
| 12:03 |
<ori> |
rebuilt wikiversions.cdb and synchronized wikiversions files: (no message) |
[production] |
| 11:32 |
<_joe_> |
rebooting mw1017 |
[production] |
| 11:29 |
<akosiaris> |
mw1130 has broken disk |
[production] |
| 11:09 |
<ori> |
running rsync-common on mw1017 |
[production] |
| 11:02 |
<hoo> |
Synchronized php-1.24wmf16/extensions/CentralAuth/: Another shot towards bug 39996 (duration: 01m 04s) |
[production] |
| 11:01 |
<hoo> |
Synchronized php-1.24wmf15/extensions/CentralAuth/: Another shot towards bug 39996 (duration: 01m 04s) |
[production] |
| 09:29 |
<_joe_> |
reimaging mw1017 aka testwiki. |
[production] |
| 06:03 |
<springle> |
ongoing schema changes: rev_content_model, rev_content_format. on terbium, osc_host.sh processes ok to kill in emergency |
[production] |
| 03:13 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Fri Aug 8 03:12:21 UTC 2014 (duration 12m 20s) |
[production] |
| 02:30 |
<LocalisationUpdate> |
completed (1.24wmf16) at 2014-08-08 02:28:39+00:00 |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.24wmf15) at 2014-08-08 02:16:13+00:00 |
[production] |
|
2014-08-07
§
|
| 19:19 |
<jgage> |
rebooting analytics1021 for kernel upgrade |
[production] |
| 18:55 |
<bblack> |
starting the process of fixing upload cache sizes, there will be periodic slim 5xx spikes... |
[production] |
| 16:32 |
<Jeff_Green> |
temporarily disabling icinga notifications for ocg100[123] ocg service check |
[production] |
| 16:09 |
<krinkle> |
Synchronized php-1.24wmf16/extensions/GlobalCssJs/GlobalCssJs.hooks.php: 4bbf4e0ed92f9a09 (duration: 00m 05s) |
[production] |
| 15:48 |
<mutante> |
zirconium - attempt to fix apache site setup manually |
[production] |
| 15:46 |
<reedy> |
Synchronized wmf-config/extension-list-labs: (no message) (duration: 00m 13s) |
[production] |
| 15:38 |
<reedy> |
Synchronized php-1.24wmf16/maintenance/findMissingFiles.php: (no message) (duration: 00m 20s) |
[production] |
| 15:37 |
<reedy> |
Synchronized php-1.24wmf15/maintenance/findMissingFiles.php: (no message) (duration: 00m 17s) |
[production] |
| 15:12 |
<reedy> |
Synchronized wmf-config/: (no message) (duration: 00m 13s) |
[production] |
| 14:43 |
<akosiaris> |
uploaded varnish_3.0.5plus~x-wm7trusty1 on apt.wikimedia.org (for usage in trusty labs machines, notably cxserver) |
[production] |
| 14:24 |
<mutante> |
shutting down elastic1018 |
[production] |
| 14:12 |
<^d> |
elastic1018: blacklisted from shard allocation since it's dead |
[production] |
| 14:05 |
<mutante> |
depooled elastic1018 - service wasnt running and signs of broken hardware (SSD) |
[production] |
| 13:57 |
<mark> |
Temporarily set max connections to swift from cp1049 backend varnish from 1000 to 2000 |
[production] |
| 13:56 |
<mutante> |
starting elasticsearch on elastic1018 |
[production] |
| 12:23 |
<hashar> |
Zuul upgraded labs branch to match production (i.e. have same version of Zuul cloner) |
[production] |
| 12:20 |
<hashar> |
restarting Zuul |
[production] |
| 11:25 |
<hoo> |
Synchronized wmf-config/InitialiseSettings.php: I53f76a35ac - No longer allow voyage 'crats to usermerge (duration: 00m 15s) |
[production] |
| 11:13 |
<akosiaris> |
removed laner@wikimedia.org entirely. It pointed to rlane@wikimedia.org which no longer exists |
[production] |
| 11:12 |
<akosiaris> |
removed rlane from root@wikimedia.org and usability@wikimedia.org |
[production] |
| 10:45 |
<mutante> |
iron, bast1001 - installed package upgrades |
[production] |