A few high-level things to know
We are a Ruby on Rails app
Ruby on Rails is a web framework heavy on conventions over configuration. All else equal, we should try to follow Rails convention. We are currently on version 7.x.x.
We cache many content pages on the edge
To decrease loading time, we use edge-caching extensively. Taking advantage of edge-caching means that we do not go all the way to the server to render every page. However this means that, on cached pages, we don't have access to helper methods like
current_user. A page is edge-cached through our CDN (Fastly) if the controller contains this line for the relevant action:
We also use server-side caching: Rails caching. If you see
<%= cache ... %>, this is code affected in production by caching.
In some situations we may want more precise content than in others. Often when we do not need a precise number, it offers an opportunity to either estimate the content or bust the cache less frequently.
- We use the
estimated_countfor a more efficient query of registered users on the home page. We have deemed that this is probably close enough.
- On posts and comment trees without recent comments, we do not asynchronously fetch the absolute latest individual reaction counts for logged-out users because this number is likely to be correct without the async call, and if it is off-by-one, we can make the choice that it is not important that it be more precise than this.
We Mostly defer scripts for usage performance improvements
To avoid blocking the initial render, we use the
We have also experimented with different techniques involving inline CSS
We attempt to reduce our bundle size
Worst technical debt
The most widespread elements of technical debt in this application reside on the frontend. We use both the "old" approach (files in the
/assets folder) and "new" approach (files in the
We also have overgrown and inconsistent CSS. This is an area we'd love to see contributions from the community.
We also have inconsistencies and issues with how we bust caching on the edge. Ideally, we could practice resource-based purging as described in the Fastly Rails docs, but we bust specific URLs via
The algorithm behind the feed
The home feed is based on a combination of recent collective posts that are cached and delivered the same to everyone in the HTML, and additional articles fetched from a PostgreSQL Full Text Search index after page load. To determine which posts a user sees, they are ranked based on the user's followed tags, followed users, and relative weights for each tag. Additional fetched articles also follow this general pattern.
Currently, the top post on the home feed, which must have a cover image, is shared among all users.
Forem uses a variation of "instant click", via InstantClick, which swaps out page content instead of making full-page requests. This approach is similar to the one used by the Rails gem
Turbolinks, but our approach is more lightweight. The library is modified to work specifically with this Rails app and does not swap out reused elements like the navigation bar or the footer. The code for this functionality is viewable in
There are a few caveats regarding this approach. Using our approach means a non-trivial amount of functionality is reloaded on page change. A similar amount of reloading occurs when using
window.InstantClick.on('change', someFunction). This results in code that looks something like this:
Abstracting and removing these caveats is a long term goal, and contribution on that front is welcome!
We use the parameter
i=i (i for internal) to indicate to the backend that we only want the "internal" version of the page (the one without the top nav and footer, etc.)
URLS and constraints
Because we use the top directory for user-generated pages, we need to be aware of some constraints.
some-forem.com/sophia could be a user, a page, an organization, or a previously banished user. We allow users to retain two redirects and should use
:moved_permanently when a user changes their username.
Because we may silently insert
?i=i on the frontend to indicate internal nav, we need to maintain that parameter if we are redirecting. We use the method
redirect_permanently_to(location) to encompass all of this behavior.
General app concepts
Articles (or posts)
Articles are the primary form of user generated content in the application. An Article has many comments and taggings through the acts-as-taggable gem, belongs to a single user (and possibly an organization), and is the core unit of content.
Collections (or series)
Although the source code refers to them as "collections" groups of articles are referred to, throughout the user interface, as "series". They represent a collection of articles relating to the same topic, indeed, a series.
Comments belong to articles or other content (they are generally polymorphic). They belong first and foremost to the user in our design, which is reflected by the URL (
/username/tag-slug), but they are present in communal areas of the application. They are threaded, but they flatten out gradually to avoid infinitely branching threads.
The user is the authorization/identity component of logging into the app. It is also the public profile/authorship/etc. belonging to the people who use the app.
While "user" is a perfectly good technical name, it is a fairly cold way to refer to humans, so we should prefer labeling people as members, or by their name/username.
Tags are used to organize user generated content. Each tag has a set of rules which are used for moderation. Each tag is a de facto community complete with community moderators.
Some tags behave as "flare," highlighting certain articles when viewed from the index page. Tags that act as "flare" are defined in the
FlareTag object. In cases of multiple flare tags, the tag displayed is determined by its hierarchy.
Listings are classified ads. They are similar to posts in some ways, but with more limitations. They are designed to be categorized into market areas. They also make use of tags.
Credits are the currency of the platform which users can use to buy listings. The functionality of credits may be expanded in the future.
Users can belong to organizations, which have their own profile pages where posts can be published etc. This can be any group endeavor such as a company, an open source project, or any standalone publication on Forem.
Hearts, unicorns, and bookmarks. Reactions are the medium for displaying appreciation for content. Bookmarks have the unique functionality of saving an article in the user's reading list.
How a user keeps track of the tags, users, or articles they care about. Follows impact a user's home feed and notifications.
Follows can have a "score" which indicates how much a user wants to see the element in their feed. Currently we only calculate these for tag follows, but it could be expanded to users. The user can set an "explicit" score, and the system also calculates an "implicit" score based on their activity.
Notes are an internal tool admins can use to leave information about things. Example: "This user was warned for spammy content".
Pages in the admin dashboard represent static pages to be served on the site. Admins are in full control to create and customize them to their needs using markdown or custom HTML. Pages are configured with a
slug and they will be served on either the
In order to ease development of custom HTML Pages in local environments the rake task
pages:sync is available. It will listen to changes made to a local HTML file and sync its contents to an existing Page in the database with the matching
This is far from a complete view of the app, but it covers a few core concepts.