|
2015-01-15
§
|
| 22:08 |
<bd808> |
restarted elasticsaerch on logstash1003; died from OOM |
[production] |
| 21:06 |
<subbu> |
deployed parsoid version 2fdf9298 |
[production] |
| 20:38 |
<ori> |
Synchronized wmf-config/InitialiseSettings.php: I250ecfceb: Switch all wikis to monolog logger (duration: 00m 05s) |
[production] |
| 20:04 |
<bd808> |
logstash redis queue backlog 384k events and climbing; likely related to the elasticsearch cluster flapping |
[production] |
| 19:57 |
<Coren> |
aborting labs filesystem move (not enough contiguous free space) and postponing until new shelf |
[production] |
| 18:59 |
<YuviPanda> |
this works? |
[production] |
| 18:23 |
<csteipp> |
deployed patches for T85349 T85850 T86711 |
[production] |
| 17:26 |
<ejegg> |
updated crm from bb05adf9279bd7a795906ca476e1850a85c21711 to d648ededf5c9fc2b0ebf989300ca2037956418e3 |
[production] |
| 16:51 |
<demon> |
Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 06s) |
[production] |
| 16:09 |
<bd808> |
Deleted 2015-12-* indices from logstash elasticsearch cluster |
[production] |
| 16:07 |
<anomie> |
Synchronized php-1.25wmf14/extensions/FlaggedRevs/api/actions/ApiReview.php: SWAT: Fix FlaggedRevs action=review for binary flagging [[gerrit:185180]] (duration: 00m 07s) |
[production] |
| 16:01 |
<bd808> |
Elasticsearch cluster for logstash has indices for events dated 2015-12-* again |
[production] |
| 15:49 |
<Jeff_Green> |
many frack host package updates and reboots |
[production] |
| 10:09 |
<aude> |
Synchronized php-1.25wmf14/extensions/Wikidata: fix noexternallanglinks bug (duration: 00m 13s) |
[production] |
| 08:24 |
<qchris> |
Ran kafka leader re-election to bring analytics1021 back into the set of leaders |
[production] |
| 08:03 |
<_joe_> |
restarted ES on logstash1002, not joining the cluster |
[production] |
| 07:47 |
<_joe_> |
restarted HHVM on mw1119, all threads stuck in a lock for HPHP::RequestInjectionData::onSessionInit |
[production] |
| 07:45 |
<ori> |
re-enabled puppet on osmium. i disabled it three hours ago to debug zhwiki key errors in memcached-serious.log. |
[production] |
| 07:19 |
<YuviPanda> |
set home of mwdeploy to /home/mwdeploy in LDAP |
[production] |
| 07:09 |
<YuviPanda> |
changed ldap mwdeploy user shell to /bin/bash to match puppet |
[production] |
| 06:21 |
<YuviPanda> |
ldap mass modification: Changing everyone with shell set to sillyshell to /bin/bash |
[production] |
| 04:56 |
<springle> |
upgrade db1051 db1056 trusty |
[production] |
| 04:35 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Jan 15 04:35:40 UTC 2015 (duration 35m 39s) |
[production] |
| 03:03 |
<tstarling> |
Synchronized wmf-config/InitialiseSettings.php: TitleBlacklist log (duration: 00m 06s) |
[production] |
| 02:32 |
<LocalisationUpdate> |
completed (1.25wmf15) at 2015-01-15 02:32:40+00:00 |
[production] |
| 02:32 |
<l10nupdate> |
Synchronized php-1.25wmf15/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 02:26 |
<springle> |
tin mw1084 sync-file failed socket error, manual sync-common |
[production] |
| 02:23 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1051 db1056 (duration: 00m 05s) |
[production] |
| 02:20 |
<LocalisationUpdate> |
completed (1.25wmf14) at 2015-01-15 02:20:40+00:00 |
[production] |
| 02:20 |
<l10nupdate> |
Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 01:53 |
<springle> |
raise mysql max_connections to 250 on virt1000. lots of nova persistent connections, little activity |
[production] |
| 01:26 |
<kaldari> |
Synchronized php-1.25wmf15/extensions/Thanks/: syncing Thanks for wmf15 (duration: 00m 05s) |
[production] |
| 01:26 |
<maxsem> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/185106 (duration: 00m 06s) |
[production] |
|
2015-01-14
§
|
| 22:57 |
<ori> |
Synchronized wmf-config/StartProfiler.php: I5bd397456: Restrict "forceprofile" to requests that set X-Wikimedia-Debug header (duration: 00m 06s) |
[production] |
| 22:47 |
<kartik> |
Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 05s) |
[production] |
| 22:40 |
<kartik> |
Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 06s) |
[production] |
| 22:31 |
<reedy> |
Synchronized php-1.25wmf14/includes/libs/virtualrest/ParsoidVirtualRESTService.php: (no message) (duration: 00m 05s) |
[production] |
| 22:18 |
<reedy> |
Synchronized php-1.25wmf15/includes/libs/virtualrest/ParsoidVirtualRESTService.php: (no message) (duration: 00m 05s) |
[production] |
| 22:18 |
<reedy> |
Synchronized php-1.25wmf15/includes/libs/virtualrest/ParsoidVirtualRESTService.php: (no message) (duration: 00m 05s) |
[production] |
| 22:13 |
<reedy> |
Synchronized wmf-config/InitialiseSettings.php: CT on testwiki (duration: 00m 06s) |
[production] |
| 22:09 |
<mutante> |
made an /a/tmp/ on fluorine to let wikidev write to for log analysis |
[production] |
| 22:08 |
<reedy> |
Synchronized wmf-config/InitialiseSettings.php: touch (duration: 00m 06s) |
[production] |
| 22:03 |
<mutante> |
apt-get clean on fluorine to get a little disk space |
[production] |
| 21:45 |
<kartik> |
Synchronized wmf-config: Enable ContentTranslation (duration: 00m 06s) |
[production] |
| 21:25 |
<subbu> |
reverted parsoid deploy to previous deployment (2cd6fefa) -- seeing a lot of dirty diffs post-deploy |
[production] |