|
2016-09-12
§
|
| 04:31 |
<halfak> |
manually killed since "python" process consuming 80% of memory and restarted uwsgi-ores service on ores-web-03 |
[ores] |
| 04:29 |
<halfak> |
restarted uwsgi-ores service on ores-web-03 |
[ores] |
| 04:20 |
<bblack> |
upgrade cp3048 to varnish 4 T131502 |
[production] |
| 04:06 |
<bblack> |
upgrade cp3047 to varnish 4 T131502 |
[production] |
| 03:51 |
<bblack> |
upgrade cp3046 to varnish 4 T131502 |
[production] |
| 03:36 |
<bblack> |
upgrade cp3045 to varnish 4 T131502 |
[production] |
| 03:21 |
<bblack> |
upgrade cp3044 to varnish 4 T131502 |
[production] |
| 03:05 |
<bblack> |
upgrade cp3039 to varnish 4 T131502 |
[production] |
| 02:49 |
<bblack> |
upgrade cp3038 to varnish 4 T131502 |
[production] |
| 02:34 |
<bblack> |
upgrade cp3037 to varnish 4 T131502 |
[production] |
| 02:29 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Sep 12 02:29:27 UTC 2016 (duration 5m 53s) |
[production] |
| 02:23 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.18) (duration: 10m 37s) |
[production] |
| 02:06 |
<ema> |
upgrade cp3036 to varnish 4 T131502 |
[production] |
| 01:28 |
<ema> |
upgrade cp3035 to varnish 4 T131502 |
[production] |
| 00:51 |
<ema> |
upgrade cp3034 to varnish 4 T131502 |
[production] |
|
2016-09-11
§
|
| 22:33 |
<aaron@tin> |
Synchronized wmf-config/CommonSettings.php: Lower wgMaxUserDBWriteDuration to 4 (duration: 00m 47s) |
[production] |
| 20:35 |
<Krenair> |
started cron service on deployment-salt02 again, seems it got killed Tue 2016-08-30 13:42:39 UTC - hopefully this will fix the puppet staleness alert |
[releng] |
| 18:45 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/309829 |
[releng] |
| 12:28 |
<addshore> |
update mediawiki |
[librarybase] |
| 11:19 |
<addshore> |
update queryservice UI to latest |
[librarybase] |
| 09:55 |
<addshore> |
install node on sparql host |
[librarybase] |
| 09:55 |
<addshore> |
remove php5 on sparql host |
[librarybase] |
| 09:54 |
<addshore> |
install php5 on sparql host |
[librarybase] |
| 08:44 |
<Amir1> |
ladsgroup@tin:~$ mwscript resetUserEmail.php --wiki=fawiki Sinasalek <email removed> |
[production] |
| 02:30 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sun Sep 11 02:30:18 UTC 2016 (duration 5m 55s) |
[production] |
| 02:24 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.18) (duration: 10m 03s) |
[production] |
|
2016-09-10
§
|
| 22:30 |
<urandom> |
T144826: Restarting Cassandra on restbase2004-b.codfw.wmnet (scrub complete, re-joining cluster) |
[production] |
| 19:10 |
<addshore> |
everything is back up and running..! :D |
[librarybase] |
| 14:38 |
<addshore> |
install npm on sparql instance |
[librarybase] |
| 13:47 |
<addshore> |
back up :) |
[librarybase] |
| 13:45 |
<addshore> |
reboot librarybase-reston-01 |
[librarybase] |
| 13:41 |
<addshore> |
librarybase-reston-01:/var/www/html/rdf# mv *.rdf /srv/ |
[librarybase] |
| 13:28 |
<addshore> |
'cp -R /var/lib/mysql/ /srv/mysql/' on reston-01 |
[librarybase] |
| 13:24 |
<addshore> |
librarybase-reston-01 looks out of space |
[librarybase] |
| 12:36 |
<urandom> |
T144826: Removing compaction rate limit, increasing compactor threads (from 10 to 20), and beginning scrub of local_group_wikipedia_T_parsoid_html.data (restbase2004-b.codfw.wmnet) |
[production] |
| 02:46 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sat Sep 10 02:46:34 UTC 2016 (duration 6m 11s) |
[production] |
| 02:40 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.18) (duration: 17m 53s) |
[production] |
| 01:06 |
<yuvipanda> |
migrate tools-k8s-etcd-01 to labvirt1012, is in state doing no io |
[tools] |
|
2016-09-09
§
|
| 20:53 |
<thcipriani> |
testing scap 3.2.5-1 on beta cluster |
[releng] |
| 19:29 |
<demon@tin> |
Synchronized wmf-config/wikitech.php: bizarro config loading (duration: 00m 46s) |
[production] |
| 19:27 |
<yuvipanda> |
reboot tools-exec-1218 and 1219 |
[tools] |
| 18:23 |
<demon@tin> |
Synchronized wmf-config/: prune old ext messages files (duration: 00m 52s) |
[production] |
| 18:10 |
<yuvipanda> |
killed massive grep running as root |
[tools] |
| 16:54 |
<demon@tin> |
Synchronized multiversion/: rm one more ugly file (duration: 01m 05s) |
[production] |
| 16:53 |
<demon@tin> |
Synchronized docroot/noc/conf/: Updating activeMWVersions data (duration: 00m 47s) |
[production] |
| 16:29 |
<legoktm@tin> |
Synchronized php-1.28.0-wmf.18/extensions/JsonConfig/: Unbreak Zero namespace, Check globals in addition to attributes https://gerrit.wikimedia.org/r/309598 (duration: 00m 51s) |
[production] |
| 16:10 |
<legoktm> |
live hacking on mw1017 |
[production] |
| 15:15 |
<urandom> |
T133805: Renabling Pupppet, forcing run, and restarting Cassandra to restore 8M region size on restbase1013-a.eqiad.wmnet |
[production] |
| 14:52 |
<Jeff_Green> |
authdns-update for pay-lvs1001 & pay-lvs1002 |
[production] |
| 14:51 |
<mobrovac> |
change-prop deployed 34b23e7 |
[production] |