|
2015-10-08
§
|
| 09:21 |
<jynus> |
downtime for db1055 for maintenance (kernel update, mysql update, config update) |
[production] |
| 08:45 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1055 (duration: 00m 17s) |
[production] |
| 07:44 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Increase load of db1051 and db1055 also for regular traffic (duration: 00m 17s) |
[production] |
| 07:06 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Matching db1051 and db1055 weight for load balancing (duration: 00m 16s) |
[production] |
| 06:15 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Oct 8 06:15:16 UTC 2015 (duration 15m 15s) |
[production] |
| 05:56 |
<ori@tin> |
Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 18s) |
[production] |
| 03:04 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.27.0-wmf.2) at 2015-10-08 03:04:52+00:00 |
[production] |
| 03:02 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.2/cache/l10n: l10nupdate for 1.27.0-wmf.2 (duration: 05m 33s) |
[production] |
| 02:46 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.27.0-wmf.1) at 2015-10-08 02:46:57+00:00 |
[production] |
| 02:40 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.1/cache/l10n: l10nupdate for 1.27.0-wmf.1 (duration: 10m 30s) |
[production] |
| 01:17 |
<twentyafterfour> |
phabricator is now up and running again |
[production] |
| 01:09 |
<twentyafterfour> |
buggy update, stopping apache2 on iridium. |
[production] |
| 01:02 |
<twentyafterfour> |
finished phabricator update |
[production] |
| 00:51 |
<mutante> |
mw1160 - was oom-killer (convert, mw job 31497) |
[production] |
| 00:50 |
<twentyafterfour> |
phabricator maintenance/upgrade. Expect 10 minutes downtime |
[production] |
| 00:44 |
<mutante> |
powercycled mw1160 - looks like broken hardware or cable (ata1: lost interrupt) |
[production] |
| 00:08 |
<krenair@tin> |
Synchronized w/static/apple-touch/commons.png: https://gerrit.wikimedia.org/r/#/c/243685/ (duration: 00m 17s) |
[production] |
|
2015-10-07
§
|
| 23:36 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/242842/ (duration: 00m 17s) |
[production] |
| 23:34 |
<ori@tin> |
Synchronized wmf-config: I924d8e19e17: Make the redis cache configuration multi-DC-ready (T111575) (duration: 00m 17s) |
[production] |
| 23:22 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/244177/ (duration: 00m 17s) |
[production] |
| 23:18 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/244176/ (duration: 00m 17s) |
[production] |
| 23:08 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/243837/ (duration: 00m 17s) |
[production] |
| 21:25 |
<ori@tin> |
Synchronized wmf-config/CommonSettings.php: Revert "Don't use nutcracker on wikitech" (duration: 00m 16s) |
[production] |
| 20:56 |
<mutante> |
applied firewalling on IRCd server, rc bot still working fine, all public IRC ports as before |
[production] |
| 20:25 |
<mutante> |
argon: installing package upgrades |
[production] |
| 20:24 |
<mutante> |
resetting drac on argon |
[production] |
| 20:14 |
<ori@tin> |
Synchronized wmf-config/session.php: Ie25c368a: Switch mw1017 to use DC-specific redis cluster names (duration: 00m 17s) |
[production] |
| 19:50 |
<urandom> |
RESTBase deploy complete |
[production] |
| 19:41 |
<urandom> |
doing full deploy of c20e6336 to RESTBase |
[production] |
| 19:29 |
<urandom> |
canary deploy to restbase1001.eqiad complete |
[production] |
| 19:14 |
<urandom> |
deploying c20e6336 to canary node restbase1001.eqiad |
[production] |
| 19:09 |
<twentyafterfour@tin> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.2 |
[production] |
| 18:56 |
<twentyafterfour@tin> |
Synchronized wmf-config/CommonSettings.php: fix undefined variable warning that has been spamming logs (duration: 00m 17s) |
[production] |
| 16:39 |
<mutante> |
powercycling analytics1035 |
[production] |
| 16:14 |
<mobrovac> |
citoid deploying ec149fd5 |
[production] |
| 16:02 |
<marktraceur@tin> |
Synchronized wmf-config/: Adding new config variable for uploads to Commons (duration: 00m 17s) |
[production] |
| 15:44 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.1/extensions/MobileFrontend: SWAT (duration: 00m 17s) |
[production] |
| 15:44 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.2/extensions/Echo: SWAT (duration: 00m 18s) |
[production] |
| 15:44 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.1/extensions/Echo: SWAT (duration: 00m 17s) |
[production] |
| 15:13 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1051 after maintenance (duration: 00m 17s) |
[production] |
| 15:03 |
<krenair@tin> |
Synchronized visualeditor-default.dblist: https://gerrit.wikimedia.org/r/#/c/242041/ (duration: 00m 17s) |
[production] |
| 15:02 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/242041/ (duration: 00m 18s) |
[production] |
| 14:59 |
<mobrovac> |
AQS restarting restbase on aqs100x |
[production] |
| 13:00 |
<godog> |
decomission restbase-test2001 and reimage |
[production] |
| 12:27 |
<paravoid> |
salt-rm'ing /var/lib/apt/lists/ubuntu.wikimedia.org_ubuntu_dists_trusty_main_i18n_Translation-en%5fUS |
[production] |
| 10:55 |
<godog> |
reenable puppet on restbase / maps-test / aqs |
[production] |
| 10:10 |
<_joe_> |
depooling cp1059 from pybal, varnish |
[production] |
| 08:44 |
<godog> |
disable puppet on restbase, maps, aqs before merging https://gerrit.wikimedia.org/r/#/c/243127 |
[production] |
| 08:05 |
<moritzm> |
installed spice security updates on labvirt* |
[production] |
| 06:01 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Wed Oct 7 06:01:01 UTC 2015 (duration 1m 0s) |
[production] |