Adds microservice to connect Islandora to FITS webservice #3

ajstanley · 2019-07-18T13:53:03Z

GitHub Issue: (link)

Other Relevant Links (Google Groups discussion, related pull requests,
Release pull requests, etc.)

What does this Pull Request do?

Downloads, configures and installs a microservice to connect to FITs webservice. The connectors have been added already to the Alpaca role.

What's new?

All new code.
It will create a new media type dynamically - each new field type coming back from the fits xml will be added to the media type on ingest.

How should this be tested?

Once this role is run you should be able to add any Original File and a new Fits Metadata media associated with the owning object.

Additional Notes:

Any additional information that you think would be helpful when reviewing this
PR.

Interested parties

@Islandora-Devops/committers

Signed-off-by: Alan Stanley <astanley@upei.ca>

dannylamb · 2019-07-29T16:52:28Z

@ajstanley Hrm.... I added your fork to the requirements.yml file for fits, but it definitely didn't install crayfits or the islandora_fits module. Let me see what's up and I'll provide some instructions if I can sort out what I need to do to get this code to execute.

ajstanley · 2019-07-29T19:17:42Z

Looks like build-fits-site.yml didn't run. Any chance your build was running an already downloaded role?

…

On Mon, Jul 29, 2019 at 1:52 PM dannylamb ***@***.***> wrote: @ajstanley <https://github.com/ajstanley> Hrm.... I added your fork to the requirements.yml file for fits, but it definitely didn't install crayfits or the islandora_fits module. Let me see what's up and I'll provide some instructions if I can sort out what I need to do to get this code to execute. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADY2J34XG6V3IMPXYACCFDQB4N43ANCNFSM4IE3JPUQ> .

-- Alan Stanley Developer and Training Specialist Agile Humanities

dannylamb · 2019-08-05T15:41:11Z

I had to add the role back into one of the plays linked in by playbook.yml. I'm testing it out now.

dannylamb · 2019-08-05T19:50:54Z

I'm getting this when the role starts doing its thang:

TASK [Islandora-Devops.fits : Install FITS] ************************************
Monday 05 August 2019  16:15:54 -0300 (0:00:00.502)       0:39:05.127 ********* 
changed: [default]

TASK [Islandora-Devops.fits : Move FITS into place] ****************************
Monday 05 August 2019  16:17:03 -0300 (0:01:08.432)       0:40:13.560 ********* 
changed: [default]

TASK [Islandora-Devops.fits : Install FITS web service] ************************
Monday 05 August 2019  16:17:03 -0300 (0:00:00.540)       0:40:14.100 ********* 
changed: [default]

TASK [Islandora-Devops.fits : Set fits.home in catalina.properties] ************
Monday 05 August 2019  16:17:06 -0300 (0:00:02.671)       0:40:16.771 ********* 
ERROR! The requested handler 'restart tomcat8' was not found in either the main handlers list nor in the listening handlers list
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

I don't even think it's gotten to your improvements yet. And that fits code is pretty stale. We removed it well over a year ago. Maybe there's a different way to restart tomcat we should be doing?

dannylamb · 2019-08-05T20:06:09Z

Maybe it's a order of operations type thing. I added the fits role to the crayfish play, but maybe it needs to be on the tomcat play? Because this encompasses apache and tomcat, it may be a little tricky to figure out what needs to go where. I think in the long run, we'd need

the fits role
a crayfits role (or just make it part of crayfish)
an islandora_fits role to download/enable the drupal module, though maybe that could get thrown into config in geerlingguy's role if we can sort out the timing issues.

Let me see if I can get this sorted out. At least it's trying to run the fits role now.

ajstanley · 2019-08-05T21:37:30Z

The restart ought to work fine - just make sure you run FITS after the tomcat install

…

On Mon, Aug 5, 2019 at 5:06 PM dannylamb ***@***.***> wrote: Maybe it's a order of operations type thing. I added the fits role to the crayfish play, but maybe it needs to be on the tomcat play? Because this encompasses apache and tomcat, it may be a little tricky to figure out what needs to go where. I think in the long run, we'd need 1. the fits role 2. a crayfits role (or just make it part of crayfish) 3. an islandora_fits role to put download/enable the drupal module, though maybe that could get thrown into config in geerlingguy's role if we can sort out the timing issues. Let me see if I can get this sorted out. At least it's trying to run the fits role now. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADY2J26KY2YTQLT33IDUGLQDCB3HANCNFSM4IE3JPUQ> .

-- Alan Stanley Developer and Training Specialist Agile Humanities

dannylamb · 2019-08-06T14:02:50Z

Ok, I added the fits role back to the tomcat role a la Islandora-Devops/islandora-playbook@cc83a0a. Unfortunately, I forgot to pull in the newest ansible-role-alpaca in the requirements file, so there was no listener with the latest. Once I've got it, I'll issue a PR to claw-playbook with what I've got and we'll go from there.

dannylamb · 2019-08-06T15:53:41Z

@ajstanley I've pushed up what I've got to a fits branch: https://github.com/Islandora-Devops/claw-playbook/compare/fits

So now everything is downloading and deploying, but I'm getting 500's from CrayFits. And I can't seem to track down the log make heads or tails of what's happening.

ajstanley · 2019-08-06T16:28:19Z

Have a look in CrayFits/var/log

…

On Tue, Aug 6, 2019 at 12:53 PM dannylamb ***@***.***> wrote: @ajstanley <https://github.com/ajstanley> I've pushed up what I've got to a fits branch: https://github.com/Islandora-Devops/claw-playbook/compare/fits So now everything is downloading and deploying, but I'm getting 500's from CrayFits. And I can't seem to track down the log make heads or tails of what's happening. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADY2JZO5KSIT7F2KXJUI43QDGNAPANCNFSM4IE3JPUQ> .

-- Alan Stanley Developer and Training Specialist Agile Humanities

ajstanley · 2019-08-06T16:31:24Z

Also http://localhost:8000/crayfits/ should show you an error page saying filename can't be empty. On Tue, Aug 6, 2019 at 1:28 PM Alan Stanley <alanjarlathstanley@gmail.com> wrote:

…

Have a look in CrayFits/var/log On Tue, Aug 6, 2019 at 12:53 PM dannylamb ***@***.***> wrote: > @ajstanley <https://github.com/ajstanley> I've pushed up what I've got > to a fits branch: > https://github.com/Islandora-Devops/claw-playbook/compare/fits > > So now everything is downloading and deploying, but I'm getting 500's > from CrayFits. And I can't seem to track down the log make heads or tails > of what's happening. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#3>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AADY2JZO5KSIT7F2KXJUI43QDGNAPANCNFSM4IE3JPUQ> > . > -- Alan Stanley Developer and Training Specialist Agile Humanities

-- Alan Stanley Developer and Training Specialist Agile Humanities

whikloj

Sorry for the delay in reviewing this @ajstanley. I tried to mark all the spots so you didn't have to do too much hunting.

whikloj · 2019-08-19T18:00:52Z

tasks/build-fits-site.yml

@@ -0,0 +1,83 @@
+- name: Build site for FITS microservice
+  blockinfile:
+    path: "/etc/apache2/conf-available/CrayFits.conf"


Should use {{ httpd_conf_directory }} to allow for changing Apache install directory.

whikloj · 2019-08-19T18:02:07Z

tasks/build-fits-site.yml

+    path: "/etc/apache2/conf-available/CrayFits.conf"
+    create: yes
+    block: |
+      Alias "/crayfits" "/var/www/html/CrayFits/public"


Should make /var/www/html/ into a configurable variable so it can be overridden like https://github.com/Islandora-Devops/ansible-role-crayfish/blob/master/defaults/main.yml#L11

whikloj · 2019-08-19T18:02:33Z

tasks/build-fits-site.yml

+    create: yes
+    block: |
+      Alias "/crayfits" "/var/www/html/CrayFits/public"
+      <Directory "/var/www/html/CrayFits/public">


Change to use new (yet to be defined) variable.

whikloj · 2019-08-19T18:02:57Z

tasks/build-fits-site.yml

+- name: Enable FITS site
+  file:
+    src: "/etc/apache2/conf-available/CrayFits.conf"
+    dest: "/etc/apache2/conf-enabled/CrayFits.conf"


Both of these lines should use {{ http_conf_directory }} as well

whikloj · 2019-08-19T18:04:44Z

tasks/build-fits-site.yml

+
+- name: Change owner on html dir
+  file:
+    path: /var/www/html


Should use your new install directory variable

whikloj · 2019-08-19T18:06:57Z

tasks/build-fits-site.yml

+- name: Install FITS Microservice from Github
+  git:
+    repo: https://github.com/roblib/CrayFits.git
+    dest: /var/www/html/CrayFits


Use yet to be created FITS install directory variable.

whikloj · 2019-08-19T18:07:17Z

tasks/build-fits-site.yml

+
+- name: Clean Directory (Failure okay)
+  file:
+    path: '/var/www/html/CrayFits/vendor/symfony/monolog-bundle'


Use yet to be created FITS install directory variable.

whikloj · 2019-08-19T18:07:23Z

tasks/build-fits-site.yml

+
+- name: Change CrayFits directory ownership, group and permissions
+  file:
+    path: /var/www/html/CrayFits


Use yet to be created FITS install directory variable.

whikloj · 2019-08-19T18:07:39Z

tasks/build-fits-site.yml

+
+- name: Open file for logging
+  file:
+    path: /var/www/html/CrayFits/var


Use yet to be created FITS install directory variable.

whikloj · 2019-08-19T18:14:47Z

tasks/build-fits-site.yml

+- name: Run Composer On Fits Microservice
+  composer:
+    command: update
+    working_dir: /var/www/html/CrayFits


Annnnnnd use yet to be created FITS install directory variable. Also as a TODO, we need to commit the lock file and then just do composer install to ensure everyone installs the same version of the dependencies.

whikloj · 2019-08-19T18:25:14Z

tasks/build-fits-site.yml

+    owner: www-data
+    group: www-data
+
+- name: Download islandora FITS module


Actually, I think this whole section can be removed. If I understand correctly this is installing the drupal-module. @dannylamb has done that here so this is unnecessary.

ajstanley · 2019-09-04T14:11:29Z

@whikloj - Whole lotta new variables, all seem to be working well. Let me know if you find anything else that needs some attention.

rosiel · 2019-09-16T12:08:34Z

@whikloj do these changes meet your requests?

whikloj · 2019-09-16T12:12:40Z

Sorry about that @ajstanley and @rosiel. I have not tested the build with these changes but this covers my original review.

seth-shaw-unlv · 2020-01-02T20:03:46Z

defaults/main.yml

 fits_install_root: /opt
 fits_install_symlink: /opt/fits
 fits_user: tomcat8
 fits_group: tomcat8
 fits_ws: yes
+crayfits_home: /var/www/html
+fits_src_url: "https://github.com/harvard-lts/fits/releases/download/1.4.1/fits-{{ fits_version }}.zip"


https://github.com/harvard-lts/fits/releases/download/1.4.1/fits-{{ fits_version }}.zip should be https://github.com/harvard-lts/fits/releases/download/{{ fits_version }}/fits-{{ fits_version }}.zip. No sense hardcoding one and not the other.

seth-shaw-unlv

Just one minor hard-coded value to update. Otherwise, 👍.

ajstanley · 2020-01-02T20:17:23Z

Done.

…

On Thu, Jan 2, 2020 at 4:10 PM Seth Shaw ***@***.***> wrote: ***@***.**** requested changes on this pull request. Just one minor hard-coded value to update. Otherwise, 👍. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADY2J3UMO2STIYG4TEESU3Q3ZC4DANCNFSM4IE3JPUQ> .

-- Alan Stanley Developer and Training Specialist Agile Humanities

seth-shaw-unlv · 2020-01-02T20:21:24Z

👏
Doing a final build before merging.

Alan Stanley added 16 commits June 28, 2019 10:29

builds microservice

5fd4c7e

new task

5d3e27d

split file

d986423

split tasks

8c527cb

changed file structure

35fd695

formatting

a5a2337

fixed unused call

a198ef8

force download

3b9a31e

updated config

66afe0d

clean directory

6618955

Signed-off-by: Alan Stanley <astanley@upei.ca>

clean directory

22ed491

clean directory

c9b2344

allow failure

c8de163

allow logging

0161215

added conditional

c2a122f

new address

a8fbdf0

whikloj suggested changes Aug 19, 2019

View reviewed changes

whikloj reviewed Aug 19, 2019

View reviewed changes

Alan Stanley added 2 commits August 20, 2019 09:26

suggestions from code review

5f27b36

fixed punctuation

f98d076

Alan Stanley added 7 commits August 20, 2019 09:47

fixed punctuation

cabac47

fixed punctuation

a20798b

Added missing variable

c3b84d3

Added variable

a856217

added variables, updated version numbers and paths

94b569f

merge conflicts

4e13a6d

updated some variables

81aeab8

Alan Stanley added 2 commits September 4, 2019 15:09

quotation marks

0cb9783

download and install module

5ec32a1

whikloj approved these changes Sep 16, 2019

View reviewed changes

Alan Stanley and others added 4 commits September 19, 2019 12:51

file ownership

b089675

Remove vagrant hard-coding

a74d745

Centos adjustments

cd33e40

merged

c27faa5

seth-shaw-unlv reviewed Jan 2, 2020

View reviewed changes

seth-shaw-unlv suggested changes Jan 2, 2020

View reviewed changes

added variable

415ec78

seth-shaw-unlv approved these changes Jan 2, 2020

View reviewed changes

seth-shaw-unlv merged commit 804aa23 into islandora-deprecated:master Jan 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds microservice to connect Islandora to FITS webservice #3

Adds microservice to connect Islandora to FITS webservice #3

ajstanley commented Jul 18, 2019

dannylamb commented Jul 29, 2019

ajstanley commented Jul 29, 2019 via email

dannylamb commented Aug 5, 2019

dannylamb commented Aug 5, 2019 •

edited

Loading

dannylamb commented Aug 5, 2019 •

edited

Loading

ajstanley commented Aug 5, 2019 via email

dannylamb commented Aug 6, 2019

dannylamb commented Aug 6, 2019

ajstanley commented Aug 6, 2019 via email

ajstanley commented Aug 6, 2019 via email

whikloj left a comment

whikloj Aug 19, 2019

whikloj Aug 19, 2019

whikloj Aug 19, 2019

whikloj Aug 19, 2019

whikloj Aug 19, 2019

whikloj Aug 19, 2019

whikloj Aug 19, 2019

whikloj Aug 19, 2019

whikloj Aug 19, 2019

whikloj Aug 19, 2019

whikloj Aug 19, 2019

ajstanley commented Sep 4, 2019

rosiel commented Sep 16, 2019

whikloj commented Sep 16, 2019

seth-shaw-unlv Jan 2, 2020

seth-shaw-unlv left a comment

ajstanley commented Jan 2, 2020 via email

seth-shaw-unlv commented Jan 2, 2020

Adds microservice to connect Islandora to FITS webservice #3

Adds microservice to connect Islandora to FITS webservice #3

Conversation

ajstanley commented Jul 18, 2019

What does this Pull Request do?

What's new?

How should this be tested?

Additional Notes:

Interested parties

dannylamb commented Jul 29, 2019

ajstanley commented Jul 29, 2019 via email

dannylamb commented Aug 5, 2019

dannylamb commented Aug 5, 2019 • edited Loading

dannylamb commented Aug 5, 2019 • edited Loading

ajstanley commented Aug 5, 2019 via email

dannylamb commented Aug 6, 2019

dannylamb commented Aug 6, 2019

ajstanley commented Aug 6, 2019 via email

ajstanley commented Aug 6, 2019 via email

whikloj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ajstanley commented Sep 4, 2019

rosiel commented Sep 16, 2019

whikloj commented Sep 16, 2019

Choose a reason for hiding this comment

seth-shaw-unlv left a comment

Choose a reason for hiding this comment

ajstanley commented Jan 2, 2020 via email

seth-shaw-unlv commented Jan 2, 2020

dannylamb commented Aug 5, 2019 •

edited

Loading

dannylamb commented Aug 5, 2019 •

edited

Loading