Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DB Schema #8

Merged
merged 56 commits into from
Nov 27, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
a8fe808
add sequelize
kratsg Oct 24, 2017
2f60e98
add mysql backend
kratsg Oct 24, 2017
b04ac48
add authenticate method for testing the connection
kratsg Oct 25, 2017
5a73763
create services/ folder. move logger and db to services
kratsg Oct 25, 2017
695427a
add sequelize-cli
kratsg Oct 25, 2017
60f69d9
const, not var
kratsg Oct 25, 2017
31fd227
add initial setup from cli
kratsg Oct 25, 2017
2bc5d09
update to point to models correctly
kratsg Oct 25, 2017
43965e7
add configs for sequelize
kratsg Oct 25, 2017
d53fb5d
remove db.js - use sequelize layout
kratsg Oct 25, 2017
0a504ab
let's test that we can connect/authenticate
kratsg Oct 25, 2017
5d33487
comment out authentication for right now
kratsg Oct 25, 2017
28fad80
update README
kratsg Oct 25, 2017
4a5bcc2
add sequelizerc file
kratsg Oct 25, 2017
b1b69d4
use a .js for configuration instead
kratsg Oct 25, 2017
996ad08
update config to js format
kratsg Oct 25, 2017
5161b54
add more documentation in README
kratsg Oct 25, 2017
971f978
add User model
kratsg Oct 25, 2017
dfe19ac
add Category model
kratsg Oct 25, 2017
a78e196
Add introspect and terminology models
kratsg Oct 25, 2017
a3687c6
add Sign model
kratsg Oct 25, 2017
8308b1e
update table names
kratsg Nov 21, 2017
1f92343
author updated
kratsg Nov 21, 2017
91507ba
minor typo in author.js
kratsg Nov 21, 2017
144ebbd
sign updated
kratsg Nov 21, 2017
2e5f037
update region
kratsg Nov 21, 2017
12b13f2
add note about inserting new region
kratsg Nov 21, 2017
ef56990
update source
kratsg Nov 21, 2017
a7e01e9
update category
kratsg Nov 21, 2017
0c92dc8
make sure through-tables are named the same
kratsg Nov 21, 2017
dd351a3
update gloss
kratsg Nov 21, 2017
407cc59
update region relationship
kratsg Nov 21, 2017
1dc448e
fix up models to use associations correctly
kratsg Nov 21, 2017
82d57b9
add insructions for db dev in README
kratsg Nov 21, 2017
03684e7
fix up typo in category_sign
kratsg Nov 21, 2017
dd2905c
ignore DS_Store for mac
kratsg Nov 22, 2017
2ac8347
add migration to create author table
kratsg Nov 22, 2017
9003200
migration to create category table
kratsg Nov 22, 2017
de17032
add gloss migration
kratsg Nov 22, 2017
bf1fe32
add region migration
kratsg Nov 22, 2017
fb126a2
add sign migration
kratsg Nov 22, 2017
7718b50
add source migration
kratsg Nov 22, 2017
a8b154d
add author relationship to signs
kratsg Nov 22, 2017
c45f98b
add region relationship to authors
kratsg Nov 22, 2017
876ffe9
add gloss_gloss relationship
kratsg Nov 22, 2017
189ac71
add gloss-source relationship
kratsg Nov 22, 2017
1cc4b87
add region-sign relationship
kratsg Nov 22, 2017
5e74acf
add category-source relationship
kratsg Nov 22, 2017
7df5770
add category-sign relationship
kratsg Nov 22, 2017
c828efd
add gloss-sign relationship
kratsg Nov 22, 2017
a95c595
add example of migration procedure for belongsToMany
kratsg Nov 22, 2017
c83fad3
typo in README for code block
kratsg Nov 22, 2017
b175d4e
code block typo
kratsg Nov 22, 2017
f736ee9
add references for more info
kratsg Nov 22, 2017
9ce6d9d
sequelize-cli as a live dependency for running migrations
kratsg Nov 23, 2017
90e1542
a composite primary key is already unique, no need to add an extra un…
kratsg Nov 23, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# mac
.DS_Store

# Logs
logs
*.log
Expand Down
8 changes: 8 additions & 0 deletions .sequelizerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
const path = require('path');

module.exports = {
'config': path.resolve('config', 'database.js'),
'models-path': path.resolve('db', 'models'),
'seeders-path': path.resolve('db', 'seeders'),
'migrations-path': path.resolve('db', 'migrations')
}
116 changes: 115 additions & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,120 @@ which uses `mocha <https://mochajs.org/>`_, `chai <http://chaijs.com/>`_, and `s
- run the ``mocha`` test suite,
- and deploy to `heroku <https://dashboard.heroku.com/>`_

Documentation
=============

Code Structure
--------------

- `.sequelizerc <.sequelizerc>`_ contains configuration for the `sequelize cli <https://github.com/sequelize/cli>`_
- `config/ <config/>`_ contains configuration files for database and other services
- `db/ <db/>`_ contains database model, seeder, and migration files
- `index.js <index.js>`_ is our server entrypoint
- `package.json <package.json>`_ is self-explanatory
- `scripts/ <scripts/>`_ contains various scripts, such as those referenced by `package.json <package.json>`_
- `services/ <services/>`_ contains code that runs various services as singletons, the exception being sequelize which is in `db/ <db/>`_
- `test/ <test/>`_ contains our tests
- `utils.js <utils.js>`_ contains some utility functions we like

Database Schema Diagram
=======================
-----------------------
`see details <documentation/SCHEMA-DIAGRAM.rst>`_

Database Management
-------------------

See `sequelize tutorial <http://docs.sequelizejs.com/manual/tutorial/migrations.html>`_ for in-depth details. Setting up the structure for the first time with the `sequelize cli <https://github.com/sequelize/cli>`_ is as easy as running::

node_modules/.bin/sequelize init

To view all possible commands, run::

node_modules/.bin/sequelize

Models and Migrations
~~~~~~~~~~~~~~~~~~~~~

To create a model, you can use the ``model:generate`` command::

node_modules/.bin/sequelize model:generate --name User --attributes firstName:string,lastName:string,email:string

which creates a ``User`` model in `db/models/ <db/models/>`_ folder and a migration file with name like ``XYZ-create-user.js`` in `db/migrations/ <db/migrations/>`_ folder. You can then run this migration::

node_modules/.bin/sequelize db:migrate

to create this new user table. Similarly, you can undo this migration with::

node_modules/.bin/sequelize db:migrate:undo

or to undo all::

node_modules/.bin/sequelize db:migrate:undo:all

or to a specific migration::

node_modules/.bin/sequelize db:migrate:undo:all --to XYZ-create-user.js

Migration Procedure Example
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Based on the way we've organized it, we are enforcing migrations for changes to our database in an atomic, consistent, testable way. To make sure this happens correctly, realize that migrations are for changing the database structure only. Client code such as validation and helper functions for existing database relationships (adding ``belongsToMany`` for example) is not part of a migration. That said, migrations will be done on a best-effort. As an example, let's say we would like to create a migration that adds a many-to-many relationship between ``gloss`` and ``sign`` models. That is, we would like ``belongsToMany`` in both directions of the relationship.

#. Run::

node_modules/.bin/sequelize model:generate --name gloss_sign --attributes glossId:integer,signId:integer

#. Remove the generated model file under ``db/models/gloss_sign.js``. We are not adding a new "model" per-se, but we are creating a new table. Technically, we should just create a migration, but it's easier to let this set up a lot of the default entries for us to make this connection work.
#. Edit the migration file that was just created, make sure the table name is ``gloss_sign`` and not ``gloss_signs`` -- as sequelize tends to pluralize automatically.
#. Continue editing by adding the following information on the foreign keys we plan to make::

glossId: {
primaryKey: true,
allowNull: false,
type: Sequelize.INTEGER,
references: {model: 'glosses', key: 'id'},
onUpdate: 'CASCADE',
onDelete: 'CASCADE'
},
signId: {
primaryKey: true,
allowNull: false,
type: Sequelize.INTEGER,
references: {model: 'signs', key: 'id'},
onUpdate: 'CASCADE',
onDelete: 'CASCADE'
},

#. Next, edit by removing the ``id`` column from the table. Since we are using a compound primary key of ``(glossId, signId)`` and will automatically cascade updates, we don't need to do any extra work to ensure that this is unique.

#. Next, edit ``db/models/sign.js`` and ``db/models/gloss.js`` to add ``belongsToMany`` relationships. For example, in ``sign.js``::

sign.associate = models => {
...
sign.belongsToMany(models.gloss, {as: "Glosses", through: "gloss_sign", foreignKey: "signId", otherKey: "glossId"});
...
};

#. Finally, run a series of migrations to ensure that we can rewind and playback with no issues::

node_modules/.bin/sequelize db:migrate
node_modules/.bin/sequelize db:migrate:undo
node_modules/.bin/sequelize db:migrate

and that's it. You've created a many-to-many relationship with migrations! Note that the migration's job here was just to create the tables according to the kind of relationship we were adding (``belongsToMany``) and we configured the ``belongsToMany`` call based exactly on the table we created (``gloss_sign``) and the foreign keys in that table (``signId``, ``glossId``).

For more information, see the following blog posts:

- https://tech.luc.id/sequelize-it-part-1-4bb752097861
- https://codeburst.io/sequelize-migrations-setting-up-associations-985d29b61ee7

Database Setup
==============

To set up the dev environment locally for MySQL, I just ran ``brew install mysql`` to install it, then::

$ mysql.server start
$ mysql -u root
mysql> GRANT ALL PRIVILEGES ON signsfive_dev.* TO 'signsfive_dev'@'localhost' IDENTIFIED BY 'signsfive_dev';

to create a ``signsfive_dev`` user with the same password and full access to the database of the same name.
24 changes: 24 additions & 0 deletions config/database.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
module.exports = {
development: {
username: 'signsfive_dev',
password: 'signsfive_dev',
database: 'signsfive_dev',
host: '127.0.0.1',
dialect: 'mysql',
operatorsAliases: false
},
test: {
username: 'database_test',
password: null,
database: 'database_test',
host: '127.0.0.1',
dialect: 'mysql',
operatorsAliases: false
},
production: {
use_env_variable: 'CLEARDB_DATABASE_URL',
dialect: 'mysql',
pool: {min: 0, max: 10, idle: 100},
operatorsAliases: false
}
}
Empty file added db/migrations/.gitkeep
Empty file.
33 changes: 33 additions & 0 deletions db/migrations/20171122013426-create-author.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable('authors', {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
email: {
type: Sequelize.STRING
},
username: {
type: Sequelize.STRING
},
auth0_id: {
type: Sequelize.STRING
},
createdAt: {
allowNull: false,
type: Sequelize.DATE
},
updatedAt: {
allowNull: false,
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.dropTable('authors');
}
};
30 changes: 30 additions & 0 deletions db/migrations/20171122014309-create-category.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable('categories', {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
name: {
type: Sequelize.STRING
},
description: {
type: Sequelize.TEXT
},
createdAt: {
allowNull: false,
type: Sequelize.DATE
},
updatedAt: {
allowNull: false,
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.dropTable('categories');
}
};
30 changes: 30 additions & 0 deletions db/migrations/20171122032604-create-gloss.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable('glosses', {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
name: {
type: Sequelize.STRING
},
description: {
type: Sequelize.TEXT
},
createdAt: {
allowNull: false,
type: Sequelize.DATE
},
updatedAt: {
allowNull: false,
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.dropTable('glosses');
}
};
30 changes: 30 additions & 0 deletions db/migrations/20171122032710-create-region.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable('regions', {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
name: {
type: Sequelize.STRING
},
loc: {
type: Sequelize.GEOMETRY('POINT')
},
createdAt: {
allowNull: false,
type: Sequelize.DATE
},
updatedAt: {
allowNull: false,
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.dropTable('regions');
}
};
28 changes: 28 additions & 0 deletions db/migrations/20171122032851-create-sign.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable('signs', {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
giphy_id: {
allowNull: false,
type: Sequelize.STRING
},
createdAt: {
allowNull: false,
type: Sequelize.DATE
},
updatedAt: {
allowNull: false,
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.dropTable('signs');
}
};
27 changes: 27 additions & 0 deletions db/migrations/20171122033056-create-source.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable('sources', {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
url: {
type: Sequelize.STRING
},
createdAt: {
allowNull: false,
type: Sequelize.DATE
},
updatedAt: {
allowNull: false,
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.dropTable('sources');
}
};
16 changes: 16 additions & 0 deletions db/migrations/20171122050235-add-author-foreign-key-to-signs.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
'use strict';

module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.addColumn('signs', 'authorId', {
type: Sequelize.INTEGER,
references: {model: 'authors', key: 'id'},
onUpdate: 'CASCADE',
onDelete: 'SET NULL'
});
},

down: (queryInterface, Sequelize) => {
return queryInterface.removeColumn('signs', 'authorId');
}
};
16 changes: 16 additions & 0 deletions db/migrations/20171122053502-add-region-foreign-key-to-authors.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
'use strict';

module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.addColumn('authors', 'regionId', {
type: Sequelize.INTEGER,
references: {model: 'regions', key: 'id'},
onUpdate: 'CASCADE',
onDelete: 'SET NULL'
});
},

down: (queryInterface, Sequelize) => {
return queryInterface.removeColumn('authors', 'regionId');
}
};
Loading