|
2014-09-26
§
|
| 23:26 |
<mutante> |
switched noc.wikimedia.org to terbium, behind misc-web |
[production] |
| 22:01 |
<bd808> |
sudo apache2ctl graceful on logstash100[123] for ldap revert |
[production] |
| 22:00 |
<bd808> |
running puppet on logstash100[123] to revert ldap change |
[production] |
| 21:56 |
<bd808> |
sudo apache2ctl graceful on logstash100[123] for ldap change |
[production] |
| 21:35 |
<andrewbogott> |
gracefulled apache on neon |
[production] |
| 21:21 |
<mutante> |
graceful'ed apache on neon |
[production] |
| 20:45 |
<ori> |
Synchronized wmf-config/CommonSettings.php: Disable HHVM beta-feature on wikidatawiki (duration: 00m 06s) |
[production] |
| 19:58 |
<awight> |
update CRM from 25159fcfc29921b08de86f12121fb292139be09d to 3e42bac8cb7f58f5e504946f4944c69ca5553e60 |
[production] |
| 19:42 |
<mutante> |
removing root's public_html from fenari - backup kept just in case |
[production] |
| 19:15 |
<AaronS> |
Deployed security patches to CentralAuth |
[production] |
| 19:09 |
<Krinkle> |
git-deploy: Deploying integration/slave-scripts 08147c42ea42e1a5eca1d29 |
[production] |
| 19:09 |
<aaron> |
Synchronized php-1.25wmf1/extensions/CentralAuth: (no message) (duration: 00m 07s) |
[production] |
| 19:06 |
<aaron> |
Synchronized php-1.24wmf22/extensions/CentralAuth: (no message) (duration: 00m 08s) |
[production] |
| 18:15 |
<Nemo_bis> |
untruncated: andrewbogott> ldap is broken for gerrit, should be working elsewhere |
[production] |
| 18:14 |
<legoktm> |
ldap is broken |
[production] |
| 18:10 |
<K4-713> |
re-enabled donations queue consumer |
[production] |
| 17:50 |
<awight> |
CRM queue consumer disabled |
[production] |
| 17:43 |
<andrewbogott> |
upgraded libgnutls26 on ytterbium |
[production] |
| 17:35 |
<andrewbogott> |
"git reset --hard origin" to remove that terrible hotfix on palladium and strontium. |
[production] |
| 17:28 |
<awight> |
CRM jobs reenabled |
[production] |
| 17:22 |
<hoo> |
Manually ran rebuildEntityPerPage.php for Wikidata |
[production] |
| 17:16 |
<andrewbogott> |
hotfixing /var/lib/git/operations/puppet in hopes of fixing gerrit so I don't have to hotfix no more |
[production] |
| 17:08 |
<awight> |
updated crm from 06c9546f9b68f6ecbaaf510944418aa52f9ed0fb to 25159fcfc29921b08de86f12121fb292139be09d |
[production] |
| 17:02 |
<awight> |
disabling CRM jobs for deployment... |
[production] |
| 15:29 |
<andrewbogott> |
puppet is now moving all labs instances to new ldap servers: ldap-eqiad and ldap-codfw |
[production] |
| 15:02 |
<cscott> |
documented what I'm going to clear the OCG queues at https://wikitech.wikimedia.org/wiki/OCG#Pruning_the_queue |
[production] |
| 14:36 |
<bblack> |
address for ns1 switched in our local dns data - https://gerrit.wikimedia.org/r/163164 |
[production] |
| 13:57 |
<hoo> |
Manually declared the global rename Secretary-> VlsergeyBot done after it twice timed out on pages moves on ruwiki |
[production] |
| 13:39 |
<akosiaris> |
moved mathoid to low-traffic lvs servers@eqiad |
[production] |
| 12:49 |
<cscott> |
cleared OCG caches again when I woke up to buy me more time to investigate the issue properly. |
[production] |
| 08:44 |
<awight> |
rollback: revision for civicrm locked to 06c9546f9b68f6ecbaaf510944418aa52f9ed0fb |
[production] |
| 08:30 |
<_joe_> |
updated hhvm on mw1053, kicked the jr a couple of times, working again now |
[production] |
| 08:29 |
<awight> |
large_donation schema migration 7000 |
[production] |
| 08:28 |
<awight> |
skip over wmf_civicrm schema migration 7022 -- *why* did I make that unsafe |
[production] |
| 08:24 |
<awight> |
fundraising_code_update: revision for civicrm changed from 06c9546f9b68f6ecbaaf510944418aa52f9ed0fb to 5aca00fd4573f0fe8f385baa7238172f6ae54438 |
[production] |
| 08:19 |
<awight> |
disabling CRM jobs during deployment |
[production] |
| 08:09 |
<cscott> |
cleared OCG queues and cache to quiet icinga; will try to get to the root cause tomorrow. |
[production] |
| 07:41 |
<hashar> |
Updated our Jenkins Job Builder fork 2d74b16..686265a |
[production] |
| 07:06 |
<ori> |
Synchronized php-1.24wmf22/extensions/WikimediaEvents: Update WikimediaEvents for 791e14cfc1d (duration: 00m 05s) |
[production] |
| 06:53 |
<ori> |
Synchronized php-1.25wmf1/extensions/WikimediaEvents: Update WikimediaEvents for 0e087daea5 (duration: 00m 07s) |
[production] |
| 06:41 |
<cscott> |
updated OCG to version f3a6c1cbba118d4a5e1aa019937dc50159fc823d |
[production] |
| 04:43 |
<_joe_> |
updating bash, USN-2363 |
[production] |
| 04:10 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Fri Sep 26 04:10:12 UTC 2014 (duration 10m 11s) |
[production] |
| 03:09 |
<LocalisationUpdate> |
completed (1.25wmf1) at 2014-09-26 03:09:47+00:00 |
[production] |
| 02:36 |
<LocalisationUpdate> |
completed (1.24wmf22) at 2014-09-26 02:36:45+00:00 |
[production] |
| 00:14 |
<awight> |
turning off Civi jobs before deployment |
[production] |