|
2014-07-09
§
|
| 23:57 |
<maxsem> |
Finished scap: SWAT, GettingStarted introduced a new message (duration: 26m 31s) |
[production] |
| 23:44 |
<mutante> |
deleted systemusers group on neon & mw1077 (to check it doesnt break anything |
[production] |
| 23:31 |
<maxsem> |
Started scap: SWAT, GettingStarted introduced a new message |
[production] |
| 23:22 |
<maxsem> |
Synchronized php-1.24wmf11/extensions/GettingStarted/: (no message) (duration: 00m 03s) |
[production] |
| 23:17 |
<maxsem> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/144857/ (duration: 00m 04s) |
[production] |
| 22:42 |
<mark> |
Enabling PAIX BGP sessions on cr2-ulsfo |
[production] |
| 22:40 |
<mark> |
Enabling WMF HQ BGP sessions on cr1-ulsfo |
[production] |
| 22:38 |
<mark> |
Enabling TiNet transit links on cr1-ulsfo |
[production] |
| 22:35 |
<mark> |
Enabling WMF HQ BGP sessions on cr2-ulsfo |
[production] |
| 22:34 |
<mark> |
Enabling NTT and HE transit links on cr2-ulsfo |
[production] |
| 22:05 |
<mutante> |
restarted apache on zirconium for config change |
[production] |
| 20:07 |
<subbu> |
deployed parsoid 1632288d |
[production] |
| 18:36 |
<yurik> |
Synchronized php-1.24wmf12/extensions/: update to JsonConfig, ZeroBanner, ZeroPortal (duration: 01m 22s) |
[production] |
| 18:29 |
<yurik> |
Synchronized php-1.24wmf11/extensions/: update to JsonConfig, ZeroBanner, ZeroPortal (duration: 01m 24s) |
[production] |
| 17:17 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: eswiki cirrus (duration: 00m 04s) |
[production] |
| 16:51 |
<csteipp> |
Finished scap: Update CentralAuth for Global Rename (duration: 28m 46s) |
[production] |
| 16:22 |
<csteipp> |
Started scap: Update CentralAuth for Global Rename |
[production] |
| 16:17 |
<mark> |
ulsfo is now offline |
[production] |
| 16:17 |
<mark> |
Shutdown NTT BGP sessions on cr2-ulsfo |
[production] |
| 16:13 |
<mark> |
Shutdown TiNet BGP sessions on cr1-ulsfo |
[production] |
| 16:10 |
<mark> |
Shutdown IXP BGP sessions on cr2-ulsfo |
[production] |
| 16:10 |
<mark> |
Shutdown WMF HQ BGP sessions on cr2-ulsfo |
[production] |
| 16:09 |
<mark> |
Shutdown WMF HQ BGP sessions on cr1-ulsfo |
[production] |
| 16:03 |
<hoo> |
Synchronized php-1.24wmf12/extensions/Wikidata/: Update Wikibase to fix a fatal and various JS things (duration: 00m 14s) |
[production] |
| 14:13 |
<hashar> |
Jenkins: bringing back puppet-compiler02.eqiad.wmflabs node online. /tmp get filled when running huge catalog compilations which causes Jenkins to unpool the node :/ |
[production] |
| 13:30 |
<godog> |
reboot ms-be1005, raid controller confused (?) after disk replacement |
[production] |
| 12:52 |
<godog> |
umounted sdg1 on ms-be1005, device disappeared, errors in dmesg |
[production] |
| 12:36 |
<bblack> |
enabled amssq47 text frontend cache in pybal for esams |
[production] |
| 09:30 |
<hashar> |
restarted Zuul to clear out stalled items in queue |
[production] |
| 09:12 |
<hashar> |
Jenkins being slow because the mediawiki-core* jobs history cache has been wiped out while updating their configuration. Jenkins is busy processing the history :( |
[production] |
| 09:02 |
<hashar> |
Jenkins killing slave process on lanthanum. Some job is stalled and unrecoverable. |
[production] |
| 08:53 |
<godog> |
upgrade ms-be1013/1014/1015 (zone5) to icehouse swift |
[production] |
| 08:51 |
<hashar> |
Jenkins migrating jobs to use $ZUUL_URL instead of git://zuul.eqiad.wmnet Preparing to scale out Zuul merger to several nodes |
[production] |
| 08:19 |
<godog> |
upgrade ms-be1009/1010/1011 (zone4) to swift icehouse |
[production] |
| 08:04 |
<hashar> |
Jenkins: granted matanya the ability to manually trigger builds. Use case: the puppet compiler! |
[production] |
| 08:02 |
<godog> |
upgrade ms-be1005/1006/1007 (zone3) to swift icehouse |
[production] |
| 03:37 |
<mutante> |
ran puppet on neon - false puppet failure alarms |
[production] |
| 02:55 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed Jul 9 02:54:37 UTC 2014 (duration 54m 36s) |
[production] |
| 02:26 |
<LocalisationUpdate> |
completed (1.24wmf12) at 2014-07-09 02:25:33+00:00 |
[production] |
| 02:15 |
<LocalisationUpdate> |
completed (1.24wmf11) at 2014-07-09 02:14:38+00:00 |
[production] |
| 01:26 |
<mutante> |
Bugzilla - enabled https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security |
[production] |
| 00:50 |
<mutante> |
restarted gitblit service |
[production] |
|
2014-07-08
§
|
| 22:50 |
<mutante> |
radon (phab)- package and kernel upgrades, rebooting |
[production] |
| 20:22 |
<legoktm> |
finished running migrateAccount.php --attachbroken --attachmissing (bug 61876) |
[production] |
| 20:07 |
<legoktm> |
finished migrateAccount.php --safe, now starting migrateAccount.php --attachbroken |
[production] |
| 20:05 |
<mutante> |
restarted apache on ytterbium |
[production] |
| 19:47 |
<K4-713> |
updated payments fraud filters again |
[production] |
| 19:47 |
<legoktm> |
running migrateAccount.php --safe for accounts only existing on one wiki (bug 39817) |
[production] |
| 19:27 |
<mutante> |
this should have fixed all the services behind misc. varnish now getting an actual "A" rating on ssllabs |
[production] |
| 19:21 |
<mutante> |
arr, i meant "nginx", not varnish |
[production] |