|
2014-06-11
§
|
| 23:41 |
<mwalker> |
Finished scap: SWAT deploy for MultimediaViewer, CentralNotice, and testwiki config (duration: 24m 16s) |
[production] |
| 23:16 |
<mwalker> |
Started scap: SWAT deploy for MultimediaViewer, CentralNotice, and testwiki config |
[production] |
| 23:10 |
<Krinkle> |
Running deleteEqualMessages.php on trwiki (bug 43917) |
[production] |
| 22:58 |
<yurik> |
Synchronized wmf-config/: Restoring to ZRMA for now (duration: 01m 04s) |
[production] |
| 22:22 |
<yurik> |
Synchronized wmf-config/InitialiseSettings.php: Attempting to enable new zero ext on zerowiki & ruwiki - take2 (duration: 01m 06s) |
[production] |
| 22:19 |
<^d> |
restarted elasticsearch on logstash1003, complaining about heap. |
[production] |
| 22:06 |
<yurik> |
Synchronized wmf-config/InitialiseSettings.php: Attempting to enable new zero ext on zerowiki & ruwiki (duration: 01m 12s) |
[production] |
| 21:58 |
<yurik> |
Synchronized php-1.24wmf8/extensions/JsonConfig/: (no message) (duration: 01m 11s) |
[production] |
| 21:56 |
<yurik> |
Synchronized php-1.24wmf7/extensions/JsonConfig/: (no message) (duration: 01m 09s) |
[production] |
| 21:50 |
<yurik> |
Finished scap: (no message) (duration: 25m 51s) |
[production] |
| 21:46 |
<ori> |
Disabling Puppet on mw1149. It's a former bits app server that isn't in PyBal so it isn't getting traffic. Going to stage some proposed changes for apache-config and operations/puppet there. |
[production] |
| 21:24 |
<yurik> |
Started scap: (no message) |
[production] |
| 21:05 |
<yurik> |
Finished scap: Deploying 3 new ext (JsonConfig, ZeroBanner, ZeroPortal), but they are not enabled anywhere yet (duration: 05m 03s) |
[production] |
| 21:00 |
<yurik> |
Started scap: Deploying 3 new ext (JsonConfig, ZeroBanner, ZeroPortal), but they are not enabled anywhere yet |
[production] |
| 20:07 |
<gwicke> |
deployed Parsoid 3de0dba15 |
[production] |
| 19:18 |
<bblack> |
rebooting lvs3003 for 3.13 kernel |
[production] |
| 19:18 |
<marktraceur> |
Finished scap: MultimediaViewer fixes for cards 630, 429, and 697 (duration: 18m 45s) |
[production] |
| 19:17 |
<greg-g> |
mw1151 *still* giving permission denied errors (publickey), what's the status, yo? |
[production] |
| 19:03 |
<bblack> |
rebooting lvs3002 for 3.13 kernel + XPS |
[production] |
| 18:59 |
<marktraceur> |
Started scap: MultimediaViewer fixes for cards 630, 429, and 697 |
[production] |
| 18:44 |
<ottomata> |
disabling puppet on analytics1012 to allow for more replica threads to catch up with current broker replicas...maybe :) |
[production] |
| 18:41 |
<awight> |
updated crm from b6815d29de97b80a0ab65db576213a604f0c7cb9 to c38296add61421f87e12cb5b4f3dd68bdf2340db |
[production] |
| 18:03 |
<Krinkle> |
Reloading Zuul to deploy I5d154a4002d08 |
[production] |
| 16:43 |
<bblack> |
shutting off lvs3002.esams pybal to test XPS balancing of live traffic on lvs3004.esams + 3.13 |
[production] |
| 16:30 |
<bblack> |
rebooting lvs3004 (inactive uploads LVS) for 3.13 again |
[production] |
| 14:52 |
<hashar> |
Jenkins restarting (plugin upgrades) |
[production] |
| 14:48 |
<bblack> |
rebooting lvs3004.esams (inactive uploads LVS) for 3.13 kernel |
[production] |
| 14:41 |
<_joe_> |
manually ran 'planet' on en.planet to restore technews |
[production] |
| 14:40 |
<hashar> |
Jenkins updating plugins |
[production] |
| 13:56 |
<paravoid> |
upgrading mw1153-mw1160, tmh1001-tmh1002 for USN-2244-1 |
[production] |
| 12:21 |
<_joe_> |
set up a secondary remote named 'readonly' in /a/common on tin, to use with the icinga check for unmerged commits |
[production] |
| 11:40 |
<akosiaris> |
manually cleaning librenms tables. db1001 is going to have increased load for some time. The approach is automatable, see http://jira.observium.org/browse/OBSERVIUM-757 |
[production] |
| 11:32 |
<godog> |
restarted uwsgi on tungsten, a lot of accesses to reqstats.edits.*.submits |
[production] |
| 10:45 |
<godog> |
restarted uwsgi on tungsten, hung on fetching many metrics |
[production] |
| 09:54 |
<_joe_> |
restarted apache on palladium - passenger crashed |
[production] |
| 05:26 |
<paravoid> |
restarting all swift daemons across the cluster to fix runaway threads due to rsyslog restart |
[production] |
| 05:04 |
<springle> |
beginning schema changes bug 49193 page_content_model |
[production] |
| 03:29 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Jun 11 03:28:14 UTC 2014 (duration 28m 13s) |
[production] |
| 02:29 |
<LocalisationUpdate> |
completed (1.24wmf8) at 2014-06-11 02:28:18+00:00 |
[production] |
| 02:15 |
<LocalisationUpdate> |
completed (1.24wmf7) at 2014-06-11 02:14:43+00:00 |
[production] |
|
2014-06-10
§
|
| 23:34 |
<andrewbogott> |
updated labs Trusty image w/puppet3, made default |
[production] |
| 23:19 |
<mutante> |
rebooting unresponsive ms-be1003 |
[production] |
| 21:10 |
<RobH> |
montly sms credit check: 1,447.36 SMS credits. will check again in 30 days |
[production] |
| 19:47 |
<hashar> |
Jenkins restarted apparently properly. Any breakage would probably be related to the version switch :-D |
[production] |
| 19:45 |
<ottomata> |
power cycling analytics1012, attempting to reinstall as kafka broker with new kafka partman recipe |
[production] |
| 19:42 |
<hashar> |
Jenkins upgraded from 1.532.2 to 1.554.2 (i.e. bumped to a new LTS version). |
[production] |
| 19:37 |
<hashar> |
Broke Jenkins by silently upgrading it :-( |
[production] |
| 19:09 |
<Krinkle> |
git-deploy: Deploying integration/slave-scripts I9521890b911714edf2 |
[production] |
| 18:59 |
<reedy> |
Synchronized php-1.24wmf8/skins/vector/components/tabs.less: (no message) (duration: 00m 14s) |
[production] |
| 18:58 |
<mutante> |
shutting down ekrem |
[production] |