|
2014-06-07
§
|
| 23:48 |
<hoo> |
Fixed four CentralAuth log entries on meta which were logged for WikiSets/0 |
[production] |
| 21:36 |
<manybubbles> |
that means I turned off puppet and shut down Elasticsearch on elastic1017 - you can expect the cluster to go yellow for half an hour or so while the other nodes take rebuild the redundency that elastic1017 had |
[production] |
| 21:35 |
<manybubbles> |
after consulting logs - elastic1017 has had high io wait since it was deployed - I'm taking it out of rotation |
[production] |
| 21:31 |
<manybubbles> |
elastic1017 is sick - thrashing to death on io - restarting Elasticsearch to see if it recovers unthrashed |
[production] |
| 17:56 |
<godog> |
restarted ES on elastic1017.eqiad.wmnet (at 17:22 UTC) |
[production] |
| 03:24 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Sat Jun 7 03:23:32 UTC 2014 (duration 23m 31s) |
[production] |
| 02:31 |
<LocalisationUpdate> |
completed (1.24wmf8) at 2014-06-07 02:29:57+00:00 |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.24wmf7) at 2014-06-07 02:16:30+00:00 |
[production] |
|
2014-06-06
§
|
| 23:51 |
<Krinkle> |
Restarted Jenkins, force stopped Zuul, started Zuul, configure Jenkins via web interface (disable Gearman, save, enable German); Seems to be back up now, finally. |
[production] |
| 22:52 |
<mutante> |
same for rhenium, titanium, bast1001, calcium, carbon, ytterbium, stat1003 |
[production] |
| 22:43 |
<RoanKattouw> |
Restarting Jenkins didn't help, jobs still aren't making it across from Zuul into Jenkins |
[production] |
| 22:36 |
<RoanKattouw> |
Restarting stuck Jenkins |
[production] |
| 22:35 |
<mutante> |
same for holmium, hafnium, silver, netmon1001, magnesium, neon, antimony |
[production] |
| 22:17 |
<mutante> |
upgraded ssl packages on zirconium |
[production] |
| 21:57 |
<Krinkle> |
Took Jenkins slave on gallium temporarily offline and back online to resolve possible stagnation |
[production] |
| 20:56 |
<awight_> |
updated crm from ded541894a70922e098fb3ea48306c8ec0f0f6aa to b38497a9d0ef75fe2b20b03b649ac13a5e3f47a7 |
[production] |
| 18:25 |
<mwalker> |
updating payments from e823354822c7a35e6c2069d3e72180a45dbc89dc to b4c5cf1bceb70d65eae28cdd0873036dc33c8992 for globalcollect oid hack |
[production] |
| 14:04 |
<hashar> |
Gerrit back. chase rebooted it :) |
[production] |
| 13:55 |
<hashar> |
Gerrit having some troubles: error: RPC failed; result=22, HTTP code = 503 (while cloning CirrusSearch ) |
[production] |
| 12:58 |
<cmjohnson1> |
replacing raid controller db1020 |
[production] |
| 06:12 |
<Tim> |
on osmium installed nodejs for testing |
[production] |
| 04:24 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Fri Jun 6 04:23:08 UTC 2014 (duration 23m 7s) |
[production] |
| 03:13 |
<LocalisationUpdate> |
completed (1.24wmf8) at 2014-06-06 03:12:19+00:00 |
[production] |
| 02:43 |
<LocalisationUpdate> |
completed (1.24wmf7) at 2014-06-06 02:42:28+00:00 |
[production] |
| 00:38 |
<bblack> |
nginx restarted on ssl* |
[production] |
| 00:16 |
<mutante> |
fixed permissions on bugzilla's index.cgi, sry |
[production] |
|
2014-06-05
§
|
| 23:18 |
<maxsem> |
Synchronized php-1.24wmf7/includes/ChangeTags.php: https://gerrit.wikimedia.org/r/#/c/137563/ (duration: 00m 03s) |
[production] |
| 23:16 |
<maxsem> |
Synchronized php-1.24wmf8/includes/ChangeTags.php: https://gerrit.wikimedia.org/r/#/c/137563/ (duration: 00m 03s) |
[production] |
| 23:06 |
<maxsem> |
Synchronized php-1.24wmf7/extensions/TemplateData: https://gerrit.wikimedia.org/r/#/c/137751/ (duration: 00m 04s) |
[production] |
| 22:15 |
<ori> |
Synchronized wmf-config/CommonSettings.php: Ife5081549: Put $wgRCFeeds[rcs100x] config behind $wmfRealm check (duration: 00m 04s) |
[production] |
| 22:12 |
<ori> |
updated /a/common to {{Gerrit|Ife5081549}}: Put $wgRCFeeds['rcs100x'] config behind $wmfRealm check |
[production] |
| 21:48 |
<ori> |
updated eventlogging to a8602c1d879f |
[production] |
| 21:34 |
<MaxSem> |
Renaming geo_killlist and geo_updates to *_old |
[production] |
| 18:36 |
<reedy> |
Synchronized wmf-config/: (no message) (duration: 00m 14s) |
[production] |
| 18:35 |
<reedy> |
Synchronized database lists: (no message) (duration: 00m 13s) |
[production] |
| 18:17 |
<Reedy> |
Created FlaggedRevs tables on ckbwiki |
[production] |
| 18:11 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Update group0 to 1.24wmf8 |
[production] |
| 18:06 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedias to 1.24wmf7 |
[production] |
| 17:00 |
<reedy> |
Synchronized wmf-config/: Wrap some long lines, add some docs (duration: 00m 26s) |
[production] |
| 16:43 |
<bblack> |
rebooting lvs3002 |
[production] |
| 16:36 |
<paravoid> |
downpref all of amslvs* in favor of lvs30* |
[production] |
| 16:17 |
<paravoid> |
downprefing amslvs1, upprefing lvs3001 |
[production] |
| 16:02 |
<mark> |
Connected cp3018:eth1 to cr1-esams:xe-0/0/3 (unconfigured) |
[production] |
| 15:59 |
<_joe_> |
disabling puppet on virt1000 while we test the puppet3 upgrade on virt0 |
[production] |
| 15:48 |
<reedy> |
Finished scap: 2nd scap for 1.24wmf8, should be effectively a nooop (duration: 12m 33s) |
[production] |
| 15:35 |
<reedy> |
Started scap: 2nd scap for 1.24wmf8, should be effectively a nooop |
[production] |