|
2015-09-19
§
|
| 18:45 |
<_joe_> |
restarted gitblit. I will now substitute myself with a clever perl one-liner. |
[production] |
| 18:38 |
<paravoid> |
pooling back cp1046 to pybal eqiad/mobile, has stayed stable |
[production] |
| 18:34 |
<paravoid> |
reactivating ΒGP with GTT @ eqiad |
[production] |
| 08:42 |
<_joe_> |
cp1046 dead on console again, powercycling to inspect it |
[production] |
| 05:49 |
<aaron@tin> |
Synchronized php-1.26wmf23/extensions/TitleBlacklist: 80d3a21a51f9c54ed2d94 (duration: 00m 12s) |
[production] |
| 05:22 |
<paravoid> |
pybal-depooling cp1046 from eqiad/mobile until further investigation |
[production] |
| 05:21 |
<paravoid> |
powercycling cp1046, dead on console |
[production] |
| 05:02 |
<wikibugs> |
Updated channels.yaml to: 4d6ce23148a4fa57c84139465beeb84361ebb6de Add releng-(.*) to catch all releng planning tags |
[tools.wikibugs] |
| 04:28 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Sat Sep 19 04:28:59 UTC 2015 (duration 28m 57s) |
[production] |
| 02:23 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.26wmf23) at 2015-09-19 02:23:29+00:00 |
[production] |
| 02:20 |
<l10nupdate@tin> |
Synchronized php-1.26wmf23/cache/l10n: l10nupdate for 1.26wmf23 (duration: 06m 05s) |
[production] |
|
2015-09-18
§
|
| 22:34 |
<greg-g> |
we missed log entries on the wikitech log from the 15th, see https://tools.wmflabs.org/sal/releng?p=0&q=&d=2015-09-18 |
[releng] |
| 22:33 |
<greg-g> |
test test |
[releng] |
| 22:18 |
<marxarelli> |
stopping eventlogging services on deployment-eventlogging02 and truncating logs |
[releng] |
| 22:09 |
<marxarelli> |
deployment-eventlogging02:/var full due to massive eventlogging_processor-server-side-0.log.1 ("(MainThread) Unable to process" errors) |
[releng] |
| 20:52 |
<ebernhardson> |
restart es on elastic1025 to disable dynamic scripting |
[production] |
| 20:34 |
<gwicke> |
dropped by_ns indexes on restbase title_revisions tables |
[production] |
| 19:54 |
<gwicke> |
finished deploy of restbase daacf4daa |
[production] |
| 19:45 |
<gwicke> |
re-enabled puppet on restbase100* |
[production] |
| 19:35 |
<gwicke> |
canary deploy of restbase daacf4daa on restbase1001; moving forward so that we can re-enable puppet over the weekend. |
[production] |
| 17:54 |
<ebernhardson@tin> |
Synchronized wmf-config/CommonSettings.php: Replace insecure es usage with usage of a plugin (duration: 00m 12s) |
[production] |
| 17:26 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/239395 |
[releng] |
| 16:41 |
<mutante> |
mailman now on 2.1.18 and jessie |
[production] |
| 16:14 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1021 |
[production] |
| 16:07 |
<paravoid> |
deactivating ΒGP with GTT @ eqiad |
[production] |
| 15:20 |
<godog> |
create restbase user on cassandra test cluster |
[production] |
| 14:55 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1020 |
[production] |
| 14:22 |
<bblack> |
committing lvs1007-1012 port/vlan changes for asw-d-eqiad (but leaving all 6 LVS ports in "disabled" state - T112781 ) |
[production] |
| 14:14 |
<bblack> |
committing lvs1007-12 port/vlan changes for asw-b-eqiad, round 3... |
[production] |
| 14:11 |
<mutante> |
sodium - stopped exim - rsyncing lists to fermium |
[production] |
| 14:10 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1019 |
[production] |
| 14:07 |
<mutante> |
stopped mailman on sodium |
[production] |
| 14:01 |
<bblack> |
rollback on asw-b-eqiad changes above |
[production] |
| 13:56 |
<bblack> |
committing eqiad lvs1007-1012 port/vlan changes for asw-b-eqiad |
[production] |
| 13:20 |
<bblack> |
committing eqiad lvs1007-12 port/vlan changes for asw-c-eqiad |
[production] |
| 13:16 |
<bblack> |
commiting eqiad lvs1007-12 port/vlan changes for asw2-a5-eqiad |
[production] |
| 13:12 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1018 |
[production] |
| 12:21 |
<godog> |
restart logstash on logstash1001, OOM in logs |
[production] |
| 11:55 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1017 |
[production] |
| 11:06 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1016 |
[production] |
| 10:28 |
<moritzm> |
restarted salt-master on palladium |
[production] |
| 09:46 |
<moritzm> |
installed openldap security updates on plutonium |
[production] |
| 09:37 |
<moritzm> |
installed openldap security updates on pollux |
[production] |
| 09:33 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1015 |
[production] |
| 09:20 |
<wikibugs> |
Updated channels.yaml to: 6a3566594f734c84b9e7d85600ae39b33ab366de releng is now Release-Engineering-Team |
[tools.wikibugs] |
| 08:22 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1014 |
[production] |
| 07:21 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1013 |
[production] |
| 06:15 |
<dcausse> |
elastic in eqiad plugin updates: restarting elastic1012 |
[production] |
| 04:37 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri Sep 18 04:37:42 UTC 2015 (duration 37m 41s) |
[production] |
| 03:11 |
<wikibugs> |
Updated channels.yaml to: f9dab0c9be84689bc24046982ef22e22d45402b7 OTRS → #wikimedia-otrs |
[tools.wikibugs] |