|
2014-02-01
§
|
| 20:20 |
<ori> |
synchronized php-1.23wmf11/extensions/FeaturedFeeds 'I8a600d37a: Reject FeedItem timestamps set too far in the future' |
[production] |
| 20:19 |
<ori> |
sync-dir: mw1054: No route to host; mw1073: Connection timed out |
[production] |
| 20:19 |
<ori> |
synchronized php-1.23wmf12/extensions/FeaturedFeeds 'I8a600d37a: Reject FeedItem timestamps set too far in the future' |
[production] |
| 12:21 |
<springle> |
synchronized wmf-config/db-eqiad.php 'db1052 increase LB' |
[production] |
| 11:26 |
<paravoid> |
restarting gdnsd on ns0/1/2 to reload maxmind geoip databases (FOSDEM geolocated esams, among others ;)) |
[production] |
| 02:25 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-02-01 02:25:34+00:00 |
[production] |
| 02:17 |
<LocalisationUpdate> |
completed (1.23wmf12) at 2014-02-01 02:17:03+00:00 |
[production] |
| 02:02 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-02-01 02:02:04+00:00 |
[production] |
|
2014-01-31
§
|
| 23:04 |
<awight_> |
updated tools to 7c4114f3db978996a460d154dcd5da2bd0ffb48d |
[production] |
| 23:03 |
<awight_> |
updated crm from 99c1702dee9ff230eabb001cd143e0e553a5e8a0 to 95665543297b4f6016e443bcd2194fa5e0f0c96f |
[production] |
| 21:47 |
<ori> |
finished scap: scap 1.23wmf12 for wikidata i18n changes (duration: 14m 26s) |
[production] |
| 21:37 |
<demon> |
synchronized php-1.23wmf12/extensions/LiquidThreads/pages/TalkpageView.php 'Fix for broken search bar' |
[production] |
| 21:36 |
<demon> |
synchronized php-1.23wmf11/extensions/LiquidThreads/pages/TalkpageView.php 'Fix for broken search bar' |
[production] |
| 21:35 |
<ori> |
started scap: scap 1.23wmf12 for wikidata i18n changes |
[production] |
| 21:29 |
<ori> |
synchronized php-1.23wmf12/extensions/Wikidata 'Iab3e51adf, I1364bfc86, I25dbed450: bug-fixes for 60670' |
[production] |
| 20:40 |
<mwalker|alt> |
updating civicrm from 600e09e35c59c3b8c127e5f3c3cf2df11db4f20e to deploy master for thank you messages |
[production] |
| 20:04 |
<ori> |
stopping sysv parsoid service on wtp1015 to test I74ba5f649 |
[production] |
| 16:56 |
<ottomata> |
rebooting analytics1003 |
[production] |
| 16:43 |
<ottomata> |
rebooting an03 to test kafka ulmits |
[production] |
| 16:22 |
<ottomata> |
reenabling puppet on cp3019 |
[production] |
| 16:01 |
<ottomata> |
stopping puppet on cp3019 to experiement with varnishkafka buffer levels |
[production] |
| 14:45 |
<ottomata> |
analytics1022 back up with higher nofile ulimit, now handling all kafka traffic, analytics1021 wont' boot |
[production] |
| 14:10 |
<ottomata> |
upping noflie open file limit on analytics1021 and analytics1022, rebooting |
[production] |
| 06:59 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1052, warm up' |
[production] |
| 06:51 |
<springle> |
synchronized wmf-config/db-eqiad.php 'db1020 full steam. depool db1011 for schema changes' |
[production] |
| 05:03 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1052 for schema changes' |
[production] |
| 04:57 |
<springle> |
synchronized wmf-config/db-eqiad.php 'repool db1020, warm up' |
[production] |
| 03:41 |
<tstarling> |
finished scap: mostly no-op, did a reset to revert the wikidata changes (duration: 05m 00s) |
[production] |
| 03:37 |
<tstarling> |
started scap: mostly no-op, did a reset to revert the wikidata changes |
[production] |
| 02:07 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-01-31 02:07:39+00:00 |
[production] |
| 02:02 |
<LocalisationUpdate> |
completed (1.23wmf12) at 2014-01-31 02:02:37+00:00 |
[production] |
| 02:02 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-01-31 02:01:50+00:00 |
[production] |
| 01:25 |
<springle> |
synchronized wmf-config/db-eqiad.php 'depool db1050 for schema changes' |
[production] |
| 00:19 |
<springle> |
xtrabackup clone db1042 to db1020 |
[production] |
| 00:19 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at 2014-01-31 00:19:28+00:00 |
[production] |
| 00:10 |
<LocalisationUpdate> |
completed (1.23wmf12) at 2014-01-31 00:10:17+00:00 |
[production] |
|
2014-01-30
§
|
| 23:40 |
<LocalisationUpdate> |
completed (1.23wmf11) at 2014-01-30 23:40:27+00:00 |
[production] |
| 23:14 |
<demon> |
synchronized wmf-config/Wikibase.php 'I50dfdc42: Enable quantity values' |
[production] |
| 22:59 |
<demon> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikidatawiki to wmf12 |
[production] |
| 22:36 |
<RobH> |
powercycling mw1061, system frozen |
[production] |
| 22:35 |
<RobH> |
rebooting frozen systems mw1057, mw1058, mw1060 |
[production] |
| 22:22 |
<RobH> |
powercycleing mw1039 as its also crashed |
[production] |
| 22:19 |
<RobH> |
mw1033 crashed, powercycling |
[production] |
| 22:14 |
<RobH> |
mw1036 crashed, unresponsive to console or ssh, rebooting |
[production] |