Skip to main content

Architecture

info

We’re currently making rapid changes to the product so our docs may be out of date. If you need help, please email yo@forem.com.

A few high-level things to know​

We are a Ruby on Rails app​

Ruby on Rails is a web framework heavy on conventions over configuration. All else equal, we should try to follow Rails convention. We are currently on version 7.x.x.

We cache many content pages on the edge​

To decrease loading time, we use edge-caching extensively. Taking advantage of edge-caching means that we do not go all the way to the server to render every page. However this means that, on cached pages, we don't have access to helper methods like current_user. A page is edge-cached through our CDN (Fastly) if the controller contains this line for the relevant action:

before_action :set_cache_control_headers

We also use server-side caching: Rails caching. If you see Rails.cache or <%= cache ... %>, this is code affected in production by caching.

Content precision​

In some situations we may want more precise content than in others. Often when we do not need a precise number, it offers an opportunity to either estimate the content or bust the cache less frequently.

Examples​

  • We use the estimated_count for a more efficient query of registered users on the home page. We have deemed that this is probably close enough.
  • On posts and comment trees without recent comments, we do not asynchronously fetch the absolute latest individual reaction counts for logged-out users because this number is likely to be correct without the async call, and if it is off-by-one, we can make the choice that it is not important that it be more precise than this.

We Mostly defer scripts for usage performance improvements​

To avoid blocking the initial render, we use the defer attribute to accelerate page renders. This practice results in a faster page load, and doesn't leave users waiting on heavy assets. However, this practice limits our ability to manipulate layout with JavaScript. As a rule, you should avoid relying on JavaScript for layout when working on Forem.

We have also experimented with different techniques involving inline CSS

We attempt to reduce our bundle size​

We use PreactJS, a lightweight alternative to ReactJS, and we try to reduce our bundle size with dynamic imports.

Worst technical debt​

The most widespread elements of technical debt in this application reside on the frontend. We use both the "old" approach (files in the /assets folder) and "new" approach (files in the app/javascript folder) for loading JavaScript into our Rails app.

We also have overgrown and inconsistent CSS. This is an area we'd love to see contributions from the community.

We also have inconsistencies and issues with how we bust caching on the edge. Ideally, we could practice resource-based purging as described in the Fastly Rails docs, but we bust specific URLs via EdgeCache::Bust#call.

The algorithm behind the feed​

The home feed is based on a combination of recent collective posts that are cached and delivered the same to everyone in the HTML, and additional articles fetched from a PostgreSQL Full Text Search index after page load. To determine which posts a user sees, they are ranked based on the user's followed tags, followed users, and relative weights for each tag. Additional fetched articles also follow this general pattern.

Currently, the top post on the home feed, which must have a cover image, is shared among all users.

Inter-page navigation​

Forem uses a variation of "instant click", via InstantClick, which swaps out page content instead of making full-page requests. This approach is similar to the one used by the Rails gem Turbolinks, but our approach is more lightweight. The library is modified to work specifically with this Rails app and does not swap out reused elements like the navigation bar or the footer. The code for this functionality is viewable in app/assets/javascripts/base.js.erb.

There are a few caveats regarding this approach. Using our approach means a non-trivial amount of functionality is reloaded on page change. A similar amount of reloading occurs when using window.InstantClick.on('change', someFunction). This results in code that looks something like this:

1 2

In some circumstances, this practice means the developer should pay special attention to the declaration of variables and functions. JavaScript may behave differently than expected.

Abstracting and removing these caveats is a long term goal, and contribution on that front is welcome!

We use the parameter i=i (i for internal) to indicate to the backend that we only want the "internal" version of the page (the one without the top nav and footer, etc.)

URLS and constraints​

Because we use the top directory for user-generated pages, we need to be aware of some constraints. some-forem.com/sophia could be a user, a page, an organization, or a previously banished user. We allow users to retain two redirects and should use :moved_permanently when a user changes their username.

Because we may silently insert ?i=i on the frontend to indicate internal nav, we need to maintain that parameter if we are redirecting. We use the method redirect_permanently_to(location) to encompass all of this behavior.

General app concepts​

Articles (or posts)​

Articles are the primary form of user generated content in the application. An Article has many comments and taggings through the acts-as-taggable gem, belongs to a single user (and possibly an organization), and is the core unit of content.

Collections (or series)​

Although the source code refers to them as "collections" groups of articles are referred to, throughout the user interface, as "series". They represent a collection of articles relating to the same topic, indeed, a series.

Comments​

Comments belong to articles or other content (they are generally polymorphic). They belong first and foremost to the user in our design, which is reflected by the URL (/username/tag-slug), but they are present in communal areas of the application. They are threaded, but they flatten out gradually to avoid infinitely branching threads.

Users​

The user is the authorization/identity component of logging into the app. It is also the public profile/authorship/etc. belonging to the people who use the app.

While "user" is a perfectly good technical name, it is a fairly cold way to refer to humans, so we should prefer labeling people as members, or by their name/username.

Tags​

Tags are used to organize user generated content. Each tag has a set of rules which are used for moderation. Each tag is a de facto community complete with community moderators.

Some tags behave as "flare," highlighting certain articles when viewed from the index page. Tags that act as "flare" are defined in the FlareTag object. In cases of multiple flare tags, the tag displayed is determined by its hierarchy.

Organizations​

Users can belong to organizations, which have their own profile pages where posts can be published etc. This can be any group endeavor such as a company, an open source project, or any standalone publication on Forem.

Reactions​

Hearts (also referred to as likes) and bookmarks. Reactions are the medium for displaying appreciation for content. Bookmarks have the unique functionality of saving an article in the user's reading list.

Follows​

How a user keeps track of the tags, users, or articles they care about. Follows impact a user's home feed and notifications.

Follows can have a "score" which indicates how much a user wants to see the element in their feed. Currently we only calculate these for tag follows, but it could be expanded to users. The user can set an "explicit" score, and the system also calculates an "implicit" score based on their activity.

Roles​

Through the "rolify" gem, users can have roles like "admin", etc. A role can also be associated with a model or a model instance. Such as "moderator of javascript tag"

Notes​

Notes are an internal tool admins can use to leave information about things. Example: "This user was warned for spammy content".

Pages​

Pages in the admin dashboard represent static pages to be served on the site. Admins are in full control to create and customize them to their needs using markdown or custom HTML. Pages are configured with a slug and they will be served on either the /page/slug or /slug path.

In order to ease development of custom HTML Pages in local environments the rake task pages:sync is available. It will listen to changes made to a local HTML file and sync its contents to an existing Page in the database with the matching slug.

Example: rake pages:sync[slug,/absolute/path/to/file.html]

This is far from a complete view of the app, but it covers a few core concepts.