Organizing your codebase
Though the Unison namespace tree can be organized however you like, this document suggests handy conventions to keep things tidy even with multiple concurrent workstreams, pull requests, library releases, and external dependencies being added and upgraded. We like conventions that can be followed without much thought and which make things easy, so you can direct your brainpower to actually writing Unison code, not figuring out how to organize it. 😀
We recommend using this exact same set of conventions for library maintainers, library contributors, and application developers. It makes it easy to follow the conventions regardless of which of these roles you are working in, and doesn't require reorganizing anything if you decide you want to publish your personal Unison repository as a library for others to build on.
🖋 This documentation is still in draft form. Please help test out these conventions and let us know how they work for you. Also any general feedback or questions are welcome! See this ticket or start a thread in Slack #alphatesting.
Here's what a namespace tree will look like that follows these conventions. This will be explained more below, and we'll also show how common workflows (like installing and upgrading libraries and opening pull requests) can be handled with a few UCM commands:
trunk/ List/ Nat/ ... README : Doc releaseNotes : Doc external/ dependency1.v7/ alice.mylib.v94/ alice.mylib.v87/ bob.somelib.trunk/ prs/ _myGreatPR _anotherNewFeature ... releases/ _v1/ _v2/ ... series/ _v1/ _v2/ ...
Directly under the namespace root, we have a
trunk (sub-)namespace (for the latest stable code), an
external namespace (for external dependencies), a
prs namespace (for pull requests in development, to be merged into
trunk or to some external dependency once ready), a
series namespace (for releases branches, forked off
trunk), and a
releases namespace, contained released versions of trunk.
🏗 The various namespaces starting with
_is a naming convention signifying an "archived namespace". A future version of Unison will make the contents of archived namespaces invisible unless you
cdinto them (see this ticket to track).
Let's now look at some common workflows:
- Installing a library
- Day-to-day development: creating and merging pull requests
- Using an unreleased library
- Creating a release
🐘 Reminder: If you ever goof anything up while doing these commands, never fear! Just use the
reflogcommand to review your local command history and switch your namespace tree back to an earlier point in history.
Installing a library is easy, just
pull into your
external namespace. Library authors will publish the exact
pull command to tell people how to fetch the latest release, but it will look something like:
.> pull https://github.com/alice/mylib:.releases._v6 .external.alice.mylib.v6
There's nothing magic happening here, this "installation" process is just including part of the remote namespace tree locally so it can be referenced by new code. Note that in this command, the release isn't put in an archived namespace, so it can be seen by the
find command, located during parsing of scratch files, and so on.
Upgrading libraries to the latest version
There's no problem with having multiple versions of a library installed at once, though sometimes you want to upgrade to the latest version of a library and delete any references to the old version. There are a couple cases you can encounter here. If
alice.mylib.v6 is a superset of
alice.mylib.v5, this is as simple as just deleting the
v6 is installed.
😎 Since references to definitions are by hash, not by name, any renaming of definitions done between
v6of a library won't break any of your code. Though we recommend reading the release notes to discover these sorts of changes, or using
diff.namespace somelib.v5 somelib.v6for an overview.
v6 replaced any definitions with new ones (using the
update command or
replace.type), these are recorded in a patch (more background on refactoring and patches in Unison) which can be applied locally. As a norm,
alice.mylib.v6.releaseNotes will cover how to apply patches to your local namespace. It will typically be commands like:
.> fork trunk prs.upgradeAliceLib .> patch external.mylib.v6.patches.v5_v6 prs.upgradeAliceLib .> merge prs.upgradeAliceLib trunk
Day-to-day development: creating and merging pull requests
Here's the basic workflow for drafting changes to
trunk. It's not much different than a typical Git workflow, where you create branches, hack on them, then merge them to
trunk when done:
.> fork .trunk .prs._myCoolPR
.> cd .prs._myCoolPRand hack away. Use
diff.namespace .trunk .prs._myCoolPRat any time to see what you've changed.
.> merge .prs._myCoolPR .trunkwhen you're done
🧐 We use an archived namespace for the PR itself so the definitions there don't pollute the namespace and are only visible if you're actively working on the PR by
cd-ing into it.
To propose changes to another Unison codebase works similarly. We'll use the Unison base library as an example:
.> pull https://github.com/unisonweb/base:.trunk _base(you can do this both for initial fetch and for subsequent updates)
.> fork _base .prs.base._mythingto create a copy of
_base. You can create multiple forks in
.prs.base, if you have multiple things you're working on.
- If you haven't already done so, set your default license for
.prs.baseto match the license of the codebase you're contributing to (for base it's the MIT license). See these instructions on how to configure just a sub-namespace with a different license.
cd .prs.base._mythingand hack away as before. At any time review what's changed between your namespace and
diff.namespace ._base .prs.base._mything.
- Push your forked version somewhere public. Suggestion: just mirror your root Unison namespace to a git repo
.> push email@example.com:mygithubname/allmyunisoncode. No need to maintain a separate Git repo for every Unison library you want to contribute to.
.prs.base._mything> pull-request.create https://github.com/unisonweb/base:.trunk https://github.com/myuser/allmyunisoncode:.prs.base._mythingand this will create some output. Copy that output to your clipboard. We don't literally use the GitHub pull request mechanism for Unison repositories, we use GitHub issues instead.
- Next, create a GitHub issue in the repo you're submitting the code to (that's right, an issue, not a GitHub PR). Many Unison repositories will have a GitHub issue template for this purpose. Make the issue title something descriptive, and for the issue body, paste in the output generated by
pull-request.createas well as some text describing the change, just like you would for any other pull request.
- Use the GitHub issue comments for coordinating the review. Once merged, the maintainer will close the issue.
This workflow also works fine even if the source and destination repository are the same, so you might use the above PR process when suggesting changes to a library that you maintain with other people.
Reviewing pull requests
We'll use this issue as an example. The issue created for the PR will have a
pr.load command to review the PR locally. We'll run that command in any empty namespace:
.review.pr55> pull-request.load firstname.lastname@example.org:unisonweb/base:.trunk email@example.com:ceedubs/unison-dev:.prs.base._genTests
.review.pr55> ls you'll see three sub-namespaces:
base (the original namespace),
head (the new namespace), and
merged (the result of
merge head base). Now you can do:
.review.pr55> diff.namespace base mergedto see what's changed. The numbered entries in the diff can be referenced by subsequent commands, so
diff.namespace base mergedmight be followed by
view 1-7to view the first 7 definitions listed there.
- You can use comments on the GitHub issue to coordinate if there's any changes you'd like the contributor to make before accepting the PR. You can also make changes directly in
.review.pr55> push firstname.lastname@example.org:unisonweb/base:.trunk mergedto push
.review.pr55> history mergedand copy the top namespace hash. That's the Unison namespace hash as of the merged PR. Then close the GitHub issue with a comment like "merged in #a9j230d9" and thank the contributor. 😀 If you ever want to go back in time, you can say
.> fork #a9j230d9 .pr55to give the
#a9j230d9namespace hash the name
.pr55in your tree.
- If you like,
.> delete.namespace .review.pr55to tidy up afterwards.
Keeping in sync with
Periodically, you can
pull the latest
.> pull https://gitub.com/unisonorama/myproject trunk
If you have in-progress PRs that you want to sync with the latest, you can bring them up to date using
.prs._myPR> merge .trunk.
Using unreleased library versions
Sometimes, you want to use some code that's only in
trunk of a library but hasn't made its way into a release. No problem. The install process looks the same, just do a
.> pull https://github.com/bob/mylib:.trunk .external.bob.mylib.trunk
As often as you like, you can re-issue the above command to fetch the latest version of the library. After doing so, you should then apply patches from the library to your local namespace. Check the project's README for information on how to do this, but typically, for a namespace,
bob.mylib.trunk, there will just be a patch called
bob.mylib.trunk.patch which can be applied with:
.> fork trunk prs.upgradeBob .> patch external.bob.mylib.trunk.patch prs.upgradeAliceLib
Assuming all is well after that
patch, you can
.> merge prs.upgradeBob trunk (and then sync that with any other PRs being drafted, as discussed in the previous section).
If you are feeling adventurous it's also possible to directly apply the patch to your in-progress PRs or even
How to create a release
Suppose you are creating
v12 of a library. The process is basically to
fork a copy of
- Before getting started, we suggest reviewing the curent patch in
.> view.patch trunk.patch. The term and type replacements listed here should generally be bugfixes or critical upgrades that you expect users of your library to make as well. You can use
delete.type-replacementto remove any entries you don't want to force on library users. See below for more.
- Fork a copy of
.> fork trunk series._v12
- Include the current dependencies in the release:
.> fork external series._v12._external. This ensures that anyone who obtains your library also receives its dependencies and the naming you had for those definitions at the time. (Would be great to minimize
_externalto just what's needed transitively by
trunk, but this could come later)
- Create / update
series._v12.releaseNotes : Doc. You can include the current namespace hash of
series._v12, a link to previous release notes (
releases._v11.releaseNotes), and if the release has a non-empty
patch, give some guidance on upgrading. Are all the edits type-preserving? If no, what sort of refactoring will users have to do?
squash series._v12 releases._v12to create the release. This squashed
releases._v12will have no history and is more efficient for users to
pullinto their codebase.
- Reset the patch and release notes in
.trunk> delete.patch patchand
.master> delete.term releaseNotes. Anyone can upgrade from a past release,
v3, by applying the patches
- If desired, you can also produce cumulative patches for skipping multiple versions. These can be published on an ad hoc basis. If publishing these, just include them in
- Optional: you can add updated instructions for fetching the release to the libraries page and also update the README on the repo's GitHub. You can also let folks know about the release via any other communication channels (Twitter, Slack, etc).
We don't recommend any fancy numbering scheme for versioning, just increment the version number. Use the
releaseNotes to convey metadata and additional nuance about each release.
Creating a bugfix release works the same way. Suppose the
v12 release has a bug. The bug has been fixed in the latest
trunk and you'd like to backport it. Just backport the fix to the
series._v12 namespace and continue with the release steps as before, but this time create
🖋 This section is in draft form and undergoing revisions for clarity. Feedback and questions are welcome! See this ticket.
Patches are collections of mappings from old hash to new hash (these entries are called "replacements"). We've seen above how these patches can be applied to upgrade a namespace using these replacements. The patches are built up via
replace.type commands which add replacements to a default patch (called "patch") that exists under each namespace in the tree. You can view this or any other patch using
.mylib.trunk> view.patch patch Edited Terms: List.frobnicate#92jajfh197 -> List.frobnicate Edited Terms: CinnamonRoll#93jg10ba -> SugarCookie Tip: To remove entries from a patch, use delete.term-replacement or delete.type-replacement, as appropriate.
replace.typecommands also take an optional patch name, if you want to build up a patch somewhere other than the default patch. This is handy for keeping logically unrelated patches separate. You can also
There are a lot of reasons you might add replacements to a patch during the course of development (see the development patches section below), but for patches published with a release, it's recommended to limit the replacements to cases where the old definition is invalid or would never be preferred the new version (generally just bugfixes for previously released definitions, performance improvements, and critical upgrades). Why do this? Because it makes things a lot easier for your users and avoids needless churn.
In other languages, where definitions are identified by name and the codebase is just mutated in-place, every change to the codebase amounts to a forced upgrade for users. We don't often think of it this way, but by updating
List.frobnicate, the old version of
List.frobnicate is effectively deleted and no one gets to reference the old definition of
List.frobnicate (unless they literally go through the Git history and bring it back somehow). Very often this is done not because the old version is wrong or should never be used, but because we don't feel like coming up with another name for the new definition and decide to just repurpose an existing name.
When this name repurposing is done for definitions that have already been released, it generates work that often isn't necessary.
In Unison, the decision to repurpose a name is completely separate from the decision to force an upgrade on users. If you want to repurpose a name like
List.frobnicate but the old definition is still valid (ask yourself "could someone reasonably still build on the old definition, or is the new definition always preferable?"), you can first delete the old definition (via
delete.term), or archive it by moving it to
_archive.y2020_m03_d24_frobnicate, timestamped with the current date. Then create the new definition for
If you've already done an
update, no problem. Just use
delete.type-replacement to remove replacements from the patch before release.
Here are a few common types of updates that won't usually be part of a release patch, but will instead be part of a development patch, covered next:
- Adding a parameter to a function to make it more generic: typically the less generic version is still perfectly valid
- Changing the order of parameters: typically the previous argument order is still perfectly valid
- Changing the type of the function: generally, when this is done, the function is actually doing something different
Release patches typically contain bugfixes, performance improvements, or critical updates (say some code depends on an external service, and that external service has a different interface now, invalidating the old code).
During development, new, unreleased definitions may get updated or refactored multiple times before making it into a release, for instance:
- An unreleased definition may have a bug that gets fixed before release. A patch records this bugfix.
- An unreleased definition may be stubbed out (using the
todofunction) then the stubbed definition later filled in. A patch records this replacement.
- Definititons may be refactored multiple times before being released, and these refactorings are recorded in a patch so all the developers on the release can easily stay up-to-date.
With some simple steps, you can make it easy to keep your release patch clean so when it comes time to make a release, the default patch in
.trunk.patch is totally clean and just includes the replacements you want all users to make in their code:
- When developing new definitions to be added to
trunk, do it in a separate namespace under
prs._myNewStuff(as discussed in the day-to-day workflows section). Within this namespace, use
replace.typeas much as you like. When you're ready to merge, just
- For other development patches, like bugfixing an unreleased definition in
trunk, or replacing a stub, use a separate named development patch for it rather than the default patch that becomes the release patch during the release process. We recommend
update .trunk.patches.devfor the default development patch. Anyone with in-progress pull requests can apply this patch to their work. It can be archived and reset periodically.
- When repurposing the name of a released definition, use the repurposing names steps covered above rather than using
- For updates to released code that all users should make (like bugfixes or performance improvements), go ahead and do an
updateof the default patch. Note that if you
fork trunk prs._somePR, then do your updates in
_somePR, when that namespace is merged back into trunk, the updates you made to the default patch will arrive there as well.