Changes between Version 21 and Version 22 of Webdev


Ignore:
Timestamp:
06/29/16 15:12:47 (8 years ago)
Author:
Sander Maijers
Comment:

Revise, remove old content and list steps toward basic Drupal setup

Legend:

Unmodified
Added
Removed
Modified
  • Webdev

    v21 v22  
    11[[PageOutline]]
    2 
    3 '''IN FLUX (April 2016)'''
    42
    53We use the [https://www.drupal.org Drupal] Content Management System (CMS) for [https://www.clarin.eu].
    64[http://poplarware.com/articles/drupal_cheat_sheet A general, brief overview of Drupal].
    75
    8 # Development workflow #
     6## Important links ##
     7
     8https://www.clarin.eu/admin/reports/status
     9
     10https://www.clarin.eu/admin/reports/dblog
     11
     12https://www.clarin.eu/admin/config/development/maintenance
     13
     14## Development workflow ##
    915
    1016Development follows a three-stage workflow:
     
    1925[[TicketQuery(col=ticket|priority|summary|owner|keywords|created|modified,status=accepted|new|assigned|reopened&component=Website&order=priority,max=10,table)]]
    2026
    21 # New plans #
     27# Conversion from legacy state to basic Drupal #
    2228
    23 [[Branding]]
     29This should be organized in the following stages.
    2430
    25 # Current issues (2016) #
    26 
    27 Below are the most pressing issues (defects, misconfigurations, etc.) at hand. These do not include functionality enhancements that are in the pipeline.
    28 
    29 ## The `clarin_eric_config` module depends on `wysiwyg` and other outdated and/or custom modules ##
    30 
    31 For WYSIWYG rich text editing, either `wysiwyg` or `ckeditor` is needed. `wysiwyg` is an extra layer of complexity that allows you to configure various alternative WYSIWYG editing libraries. However, only `CKEditor` seems current and still maintained among these. The CKEditor JavaScript library must be installed locally inside the Drupal code base for it to be detected by `wysiwyg`. `wysiwyg` conflicts with `ckeditor` and only supports CKEditor >= 3, < 4. After the emergency (security) update of Drupal from 7.39 to 7.43, WYSIWYG editing stopped working. For some reason, the CKEditor 4 library is/was still inside the Drupal code base. `wysiwyg` now fails to detect this CKEditor library, and this is [https://www.drupal.org/node/1853550 on purpose], since its version is not supported apparently. CKEditor 4 has been installed via the `ckeditor` module after that in order to replace `wysiwyg`. But `wysiwyg` does not work when `ckeditor` is installed. Yet `wysiwyg` is required (depended on) by the custom `clarin_eric_config` module that is currently unmaintained but essential to site functionality. Thus `ckeditor` had to be disabled againg and there is no WYSIWYG editor available anymore for the time being. The solution is to update the `clarin_eric_config` to not depend on `wysiwyg` or any other unnecessary modules, and then to replace the module `wysiwyg` with `ckeditor`.
    32 
    33 ## Update each module piece by piece, not just for security issues ##
    34 
    35 We should use only stable versions of third party modules. Furthermore, we should cut down the number of modules insofar possible.
    36 
    37 Current problems:
    38 a. Development versions (betas and even alphas) of modules are currently used in production.
    39 a. These versions are largely outdated according to `drush`.
    40 a. In addition, a lot of modules cannot be updated by `drush`. They were customized or are incompatible/unavailable for other reasons.
    41 a. It unclear whether updating the modules would break anything.
    42 
    43 # Historical situation #
    44 
    45 ## Drush availability ##
    46 
    47 Drush is installed in Bas van der Veen and Jurriaan Roelof's home directories instead of in a system-wide `$PATH` directory.
    48 Users from the Unix group `clarinadmin` can access it. At this moment Bas van der Veen can perform Drush commands (in directory `/hum/web/www.clarin.eu/htdocs/sites/clarin.eu` or `/hum/web/tst.clarin.eu/htdocs`)
    49 `/home/bvdveen/drush/drush status`
    50 
    51 The current version is outdated (5.8) and should be updated after planned OS upgrade (Feb 2016). Recent `drush` subcommands may not work otherwise.
    52 
    53 In the following, let `"${Drupal_root_dir_path}"` be the root directory path of the Drupal installation.
    54 
    55 ## Backups ##
    56 ICT & Media system administrators are responsible for a regular backup through a filesystem(?) snapshot.
    57 
    58 ### Manually, through web interface ###
    59 [https://www.clarin.eu/admin/config/system/backup_migrate]
    60 
    61 ### Manually, from command line ###
    62 With [http://www.drushcommands.com/drush-6x/archive/archive-dump `archive-dump`]
    63 {{{
    64 #!sh
    65 
    66 drush archive-dump # --no-core
    67 }}}
    68 
    69 [http://www.drushcommands.com/drush-6x/backup-migrate/bam-backup or:]
    70 {{{
    71 #!sh
    72 
    73 drush bam
    74 }}}
    75 
    76 ### Manually, from command line (error prone version) ###
    77 {{{
    78 #!sh
    79 
    80 sudo tar -cvJ -C '/hum/web/www.clarin.eu/htdocs' -f ‘bckup_clarin_28oct15’ .
    81 }}}
    82 
    83 ## Which updates are available? ##
     31## Enable Bootstrap module ##
    8432
    8533{{{
    8634#!sh
    8735
    88 drush -r "${Drupal_root_dir_path}" -s core-status Drupal
     36## Enable Bootstrap
     37drush.phar pm-enable 'jquery_update' 'bootstrap'
     38
     39## The frontpage view was disabled in the legacy state, reenable it.
     40drush.phar views-enable 'frontpage'
    8941}}}
    9042
    91 Check if a module is listed under `../sites/all/patches`. Patches are often integrated in higher non-patch versions of modules. See: [https://www.drupal.org/patch] (manually apply patch: [https://www.drupal.org/node/534548]).
    92 ----
     431. [https://dev-www-3.clarin.eu/admin/appearance ‘Sub Seven’ admin theme -> ‘Default theme’].
     442. [https://dev-www-3.clarin.eu/admin/appearance Set Bootstrap theme as default].
     453. [https://dev-www-3.clarin.eu/admin/appearance Disable ‘Clarin.eu Theme’].
     465. [https://dev-www-3.clarin.eu/admin/structure/menu/settings ‘Source for the Secondary links’: User menu -> Secondary menu].
     476. [https://dev-www-3.clarin.eu/admin/structure/block/manage/system/main-menu/configure?destination=admin/structure/menu/settings Disable block ‘Main menu’ by setting it to `- None -` in the dropdown in ‘Region settings’].
     487. [https://dev-www-3.clarin.eu/admin/config/system/site-information -> Set ‘Default front page’ to 'frontpage'].
     49
     50## Disable legacy/redundant extensions ##
    9351
    9452{{{
    9553#!sh
    9654
    97 # Or: '/hum/web/tst.clarin.eu/htdocs/'
    98 cd '/hum/web/www.clarin.eu/htdocs/sites/clarin.eu/'
    99 drush status
    100 # List patches
    101 drush ups --security-only
    102 # Apply patches
    103 drush upc --security-only
     55## Must be updated to dev version first, since otherwise it won't uninstall due to bugs.
     56drush.phar pm-enable 'empty_page-7.x-1.x-dev'
     57## IMPORTANT: Quit endless loop here!
     58
     59## Alpha.
     60drush.phar pm-disable 'uuid_features'
     61
     62## ‘Minimally maintained’.
     63drush.phar pm-disable 'smart_trim'
     64
     65## 7.x-3.0-rc5.
     66drush.phar pm-disable 'admin_menu' 'admin_menu_toolbar'
     67
     68## Apparently not in use.
     69drush.phar pm-disable 'entityreference_prepopulate' 'fences'
     70
     71## Seems unused, only used for twitter_pull_bean?
     72drush.phar pm-disable 'bean_admin_ui' 'bean' 'bean_entitycache' 'bean_uuid'
     73
     74## Functionality seems unrequired.
     75drush.phar pm-disable 'imagecache_actions' 'imagecache_canvasactions' 'empty_page' 'timeago'
     76
     77## Superfluous functionality and/or security risk.
     78drush.phar pm-disable 'backup_migrate' 'environment_indicator'  'admin_devel'
     79
     80## The following modules could be directly traced to Jurriaan. They have cyclic dependencies and define a number of features and views.
     81drush.phar pm-disable 'menu_block' 'bean_twitter_pull' 'twitter_pull' 'openscience_blog' 'openscience_events' 'openscience_groups' \
     82                      'context' 'context_ui' 'context_menu_block' 'context_og' 'clarin_eric_config' 'clarineric' 'clarin_eric_config' \
     83                      'openscience_news' 'openscience_showcases' 'lrt_showcases' 'openscience_ui' 'openscience' 'openscience_governance' \
     84                     
     85
     86## Mmake sure that the remaining extensions are up-to-date.
     87drush.phar pm-update --no-backup
    10488}}}
    10589
    106 ## Common error messages ##
     90Remove residual Drupal _views_ by visiting:
    10791
    108 ### Wrong directory ###
     921. https://dev-www-3.clarin.eu/admin/structure/views/view/openscience_events/delete
     932. https://dev-www-3.clarin.eu/admin/structure/views/view/openscience_group_river/delete
     943. https://dev-www-3.clarin.eu/admin/structure/views/view/openscience_movies/delete
     954. https://dev-www-3.clarin.eu/admin/structure/views/view/openscience_popular_content/delete
     965. https://dev-www-3.clarin.eu/admin/structure/views/view/featured_lrt_showcase/delete
     976. https://dev-www-3.clarin.eu/admin/structure/views/view/featured_showcase/delete
     987. https://dev-www-3.clarin.eu/admin/structure/views/view/lrtshowcases/delete
    10999
    110 {{{
    111 Command pm-refresh needs a higher bootstrap level to run - you will need to invoke drush from a more         [error]
    112 functional Drupal environment to run this command.
    113 }}}
     100## Enhance functionality before testing ##
    114101
    115 ### `ups` failed ###
    116 {{{
    117 The drush command 'ups' could not be found.  Run `drush cache-clear drush` to clear the commandfile cache if [error]
    118 you have installed new extensions.
    119 }}}
     102Some Website Trac tickets (at the top of this page) are feature requests/enhancements. Ideally these can be added without much risk and then included in the acceptance tests.
    120103
    121 You did not use Drush 5 or higher. Instead, you could issue:
    122 {{{
    123 #!sh
    124 
    125 drush up --security-only
    126 }}}
    127 
    128 ### Forbidden to download drupal core ###
    129 
    130 {{{
    131 It's forbidden to download drupal core into an existing core.
    132 }}}
     104https://trac.clarin.eu/ticket/938
    133105
    134106{{{
    135107#!sh
    136108
    137 drush rf
     109drush.phar cache-clear all
    138110
    139 # In case database migration is needed:
    140 drush updb                             
     111## Enable CKEditor for rich text edition (WYSIWYG).
     112drush.phar pm-enable 'ckeditor'
    141113
    142 # Should not show available updates or errors:
    143 drush status                           
     114## Enable smtp module so that Centre registration form, which depends on e-mail functionality, works.
     115drush.phar pm-enable 'smtp'
    144116}}}
    145117
    146 * Check website, and disable maintenance mode.
    147 * Possibly, Drupal Features must be reset:
     1181. [https://dev-www-3.clarin.eu/admin/config/system/smtp -> Disable Drupal queue]
     119
     120### Content restructuring ###
     121
     122The office restructures content so that excess hierarchy is flattened.
     123TODO: ...
     124
     125### Acceptance tests ###
     126
     1271. Adding a basic page.
     1282. Uploading multiple files in one go.
     1293. Writing a blog post.
     1304. Filling out the Centre Registry form.
     1315. TODO: ...
     132
     133#### Criteria ####
     134
     135Does the rich text editing work as expected?
     136
     137### Migration ###
     138
     139### Thorough cleanup ###
     140
    148141{{{
    149142#!sh
    150143
    151 drush features-list
    152 # Reset Drupal Features:
    153 drush fr -y openscience_demo_content openscience_events --force
     144## Jurriaan's modules all have cyclic dependencies, so cannot be uninstalled.
     145## IMPORTANT: The following command
     146drush.phar pm-uninstall $(drush.phar pm-list --status='disabled' --format=list --no-core)
     147## May cause Drush defects reported as:
     148## Error: require_once(): Failed opening required '/var/www/localhost/htdocs/sites/all/modules/contrib/menu_block/plugins/content_types/menu_tree/menu_tree.inc'
     149## (include_path='.:/usr/share/pear') in /var/www/localhost/htdocs/sites/all/modules/contrib/ctools/includes/plugins.inc, line 477
     150
     151## Uninstalling extensions using Drush does not result in actual removal.
     152## To forcefully remove ‘disabled, uninstalled’ modules/themes.
     153drush.phar pm-list --status='disabled,not installed' --no-core --pipe | \
     154    xargs -n 1 -I '{}' drush.phar drupal-directory --local-only --component='path' '{}' | \
     155        xargs -n 1 -I '{}' rm -frv -- '{}'
     156
     157## Clean up informal dependencies, e.g. in 'sites/all/libraries/'
     158# rm -r 'sites/all/libraries/timeago/'
     159
     160drush.phar cache-clear all
     161
     162## Remove security sensitive and redundant files from Drupal installation.
     163rm 'install.php' 'CHANGELOG.txt' 'INSTALL.txt' 'INSTALL.mysql.txt' \
     164        'INSTALL.pgsql.txt' 'LICENSE.txt' 'MAINTAINERS.txt' 'UPGRADE.txt' \
     165        'COPYRIGHT.txt' 'README.txt' 'INSTALL.sqlite.txt'
     166
     167## TODO: Remove residual Drupal Features. Cannot be done with Drush or the Drupal web interface alone.
     168...
    154169}}}
    155 * In case the previous has failed, indicated by the status being `overridden`, consult [https://www.drupal.org/node/986932]. A quick fix is to remove and reinstall all Feature modules, but for custom Features (`openscience`, `clarin_eric_config`) that will fail.
    156170
    157 ### Failed Drupal upgade ... ###
     171### Acceptance test ###
    158172
    159 [https://github.com/C13L0/roll-your-own#get-an-old-site-up-and-running Restore from backup archive].
     173### Finishing touch ###
     174
     175#### Enhancement: do not use CDN. ####
     176
     177DO NOT DO WITHOUT ALTERNATIVE:
     178https://dev-www-2.clarin.eu/admin/appearance/settings/bootstrap Make independent from `jsdeliver`: -> Advanced]
     179
     180#### Clean up content types ####
     181
     182Merge https://dev-www-3.clarin.eu/admin/content nodes of LRT Showcase type with Showcase type.
     183