| 
      
        2019-02-21
      
      §
     | 
  
    
  | 10:29 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid finished | 
  [production] | 
            
  | 10:29 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid cluster staging completed | 
  [production] | 
            
  | 10:29 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid upgrade -f mathoid-staging-values.yaml staging stable/mathoid [namespace: mathoid, clusters: staging] | 
  [production] | 
            
  | 10:28 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid finished | 
  [production] | 
            
  | 10:28 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid cluster staging completed | 
  [production] | 
            
  | 10:28 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid upgrade -f mathoid-values.yaml staging stable/mathoid [namespace: mathoid, clusters: staging] | 
  [production] | 
            
  | 10:27 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid upgrade -f mathoid-values.yaml stable/mathoid [namespace: mathoid, clusters: staging] | 
  [production] | 
            
  | 10:26 | 
  <akosiaris@deploy1001> | 
  scap-helm list finished | 
  [production] | 
            
  | 10:26 | 
  <akosiaris@deploy1001> | 
  scap-helm list cluster codfw completed | 
  [production] | 
            
  | 10:26 | 
  <akosiaris@deploy1001> | 
  scap-helm list cluster eqiad completed | 
  [production] | 
            
  | 10:26 | 
  <akosiaris@deploy1001> | 
  scap-helm list  [namespace: list, clusters: eqiad,codfw] | 
  [production] | 
            
  | 10:23 | 
  <godog> | 
  on boron unblock trusty builds with umount /var/cache/pbuilder/base-trusty-amd64.cow/dev/ptmx | 
  [production] | 
            
  | 10:04 | 
  <akosiaris> | 
  create citoid namespace on kubernetes eqiad codfw staging clusters T213194 | 
  [production] | 
            
  | 10:04 | 
  <akosiaris> | 
  create cxserver namespace on kubernetes eqiad codfw staging clusters T213195 | 
  [production] | 
            
  | 09:35 | 
  <volans> | 
  force rebooting unresponsive icinga1001 T214760 | 
  [production] | 
            
  | 09:29 | 
  <marostegui> | 
  Deploy schema change on s3 primary master (db1078) - T210713 | 
  [production] | 
            
  | 09:29 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1075 T210713 (duration: 00m 52s) | 
  [production] | 
            
  | 09:14 | 
  <moritzm> | 
  temporarily stop prometheus@labs.service on labmon for journald restarts (part of security update) | 
  [production] | 
            
  | 08:40 | 
  <marostegui> | 
  Deploy schema change on db1075 - T210713 | 
  [production] | 
            
  | 08:40 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1075 T210713 (duration: 00m 54s) | 
  [production] | 
            
  | 08:35 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1077 T210713 (duration: 00m 53s) | 
  [production] | 
            
  | 07:44 | 
  <moritzm> | 
  rolling out remaining systemd security updates on jessie | 
  [production] | 
            
  | 07:12 | 
  <marostegui> | 
  Deploy schema change on db1077 - this will generate lag on labsdb:s3 T210713 | 
  [production] | 
            
  | 07:12 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1077 T210713 (duration: 00m 56s) | 
  [production] | 
            
  | 07:08 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1123 T210713 (duration: 00m 55s) | 
  [production] | 
            
  | 06:22 | 
  <marostegui> | 
  Deploy schema change on db1123 - T210713 | 
  [production] | 
            
  | 06:21 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1123 T210713 (duration: 00m 57s) | 
  [production] | 
            
  | 05:46 | 
  <bblack> | 
  repooling cp5010 - T214274 | 
  [production] | 
            
  | 05:41 | 
  <bblack> | 
  removing cp5010 downtimes from icinga - T214274 | 
  [production] | 
            
  | 05:34 | 
  <bblack> | 
  rebooting cp5010 for device name on swapped disk (depooled) - T214274 | 
  [production] | 
            
  | 04:30 | 
  <kart_> | 
  Finished: Fifth manual run of unpublished draft purge script for ContentTranslation (T216470) | 
  [production] | 
            
  | 04:16 | 
  <XioNoX> | 
  Unplug Tata/NTT/PCCW from cr1-eqsin - T213121 | 
  [production] | 
            
  | 03:21 | 
  <XioNoX> | 
  replace cp5010 disk 1 - T214274 | 
  [production] | 
            
  | 03:15 | 
  <kart_> | 
  Fifth manual run of unpublished draft purge script for ContentTranslation (T216470) | 
  [production] | 
            
  | 02:44 | 
  <XioNoX> | 
  depool eqsin - T213121 | 
  [production] | 
            
  | 02:31 | 
  <twentyafterfour> | 
  phabricator upgrade finished, service appears to be returned to normal | 
  [production] | 
            
  | 01:43 | 
  <twentyafterfour> | 
  running phabricator database schema changes | 
  [production] | 
            
  | 01:38 | 
  <twentyafterfour> | 
  now taking phabricator offline for upgrade | 
  [production] | 
            
  | 01:15 | 
  <twentyafterfour> | 
  Taking phabricator offline momentarily for upgrade | 
  [production] | 
            
  | 01:01 | 
  <twentyafterfour> | 
  set downtime in icinga for phab100* | 
  [production] | 
            
  | 00:17 | 
  <catrope@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable partial blocks on metawiki and mediawikiwiki (T216065) (duration: 00m 54s) | 
  [production] | 
            
  
    | 
      
        2019-02-20
      
      §
     | 
  
    
  | 23:59 | 
  <ppchelko@deploy1001> | 
  Finished deploy [changeprop/deploy@5e4486a]: Purge varnish on revision restrictions (duration: 01m 23s) | 
  [production] | 
            
  | 23:57 | 
  <ppchelko@deploy1001> | 
  Started deploy [changeprop/deploy@5e4486a]: Purge varnish on revision restrictions | 
  [production] | 
            
  | 21:48 | 
  <eileen> | 
  civicrm revision changed from 165fbf5894 to 1b5d974569, config revision is ccefa3716b | 
  [production] | 
            
  | 21:46 | 
  <arlolra> | 
  Updated Parsoid to 9b204a0 (T153080, T169975, T215824) | 
  [production] | 
            
  | 21:28 | 
  <arlolra@deploy1001> | 
  Finished deploy [parsoid/deploy@c4574d1]: Updating Parsoid to 9b204a0 (duration: 09m 33s) | 
  [production] | 
            
  | 21:19 | 
  <arlolra@deploy1001> | 
  Started deploy [parsoid/deploy@c4574d1]: Updating Parsoid to 9b204a0 | 
  [production] | 
            
  | 21:08 | 
  <_joe_> | 
  rolling restart of php-fpm to catch up with the tideways change | 
  [production] | 
            
  | 20:35 | 
  <thcipriani@deploy1001> | 
  Synchronized php: group1 wikis to 1.33.0-wmf.18 (duration: 00m 53s) | 
  [production] | 
            
  | 20:33 | 
  <thcipriani@deploy1001> | 
  rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.18 | 
  [production] |