8

Ok, suppose we got it settled that both the frontend and the backend must validate input.

But how would you avoid duplication of validation logic across several application layers? The problem is you'd have to keep them in sync every time the logic changes.

One idea is to have a DB table type|regex (it would be the single source of truth), but it would cover only simple format validation, e.g. emails, URLs.

Note the question doesn't ask whether validation belongs to the presentation or domain layer.

Context

Our app is a white-label payment gateway. Our clients are payment service providers (PSPs) that don't want to actually write and maintain any gateways. Basically, they look for merchants and resell our white-label solution with their logo.

We have a frontend web client (probably, React, haven't asked, just joined the team) and a backend in Java. The user can create PSP terminals in the GUI and configure them. Configuration parameters are dynamic, that is each terminal may have its own set of parameters. There's a DB table for parameters. It has columns "name", "regex", among others, but not "type". Each time you insert a new URL parameter into the table, you're supposed to copy-paste the URL regex, i.e. introduce duplication. It doesn't look like the best solution.

So I suggested adding a new column, "type". My initial idea was to simply pass it to the frontend and let it handle format validation. But I guess it has to be more complex than that, considering the backend validation that should also be in place. Simply duplicating validation logic would defeat the original goal which was to avoid duplication in the first place. Duplication across multiple app layers would arguably be even worse than duplication of a regex string in one SQL script.

Improve this question
2
  • ... note however , the top answers of the questions where you say they are "no duplicate" apply both very much to your question, so you should really take the time and read them both. Commented Jun 28 at 11:17
  • 1
    It all depends on how complicated those checks can be. For example in my current company we store some rules as webassembly code in database. Because it can be arbitrary plus it can come from users. This of course comes with all sort of other problems (you need some compiler, runtime, you need to care about security). In other company we kept a custom pseudo language that would apply and/or/not boolean operators to some fixed identifiers. And we would keep such expressions in db as well. Less powerful, but easier to maintain. So it all depends. Commented Jun 28 at 15:22

8 Answers 8

Reset to default
10

You wrote

Duplication across multiple app layers would arguably be even worse than duplication of a regex string in one SQL script.

which I see differently. How bad some duplication is, does not so much depend on the layer where it happens, it depends on what kind of logic exactly is duplicated. But give me a little bit of space to explain this.

Before automatically assuming that any kind of logic duplication must be eliminated, you should first ask yourself what you're actually trying to achieve with it. Certain forms of duplication are perfectly tolerable, others are not. The DRY principle isn't an end in itself, it is a means to an end, and especially across layers following DRY without further considerations may lead to a worse architecture than accepting some duplication.

The goal of not duplicating logic is to avoid bugs which can occur by duplicated logic getting accidentally out of sync. Still, when it comes to validation logic in frontend and backend, non-identical validations do not necessarily introduce bugs. Validation logic in the frontend has mainly the purpose of speeding up the application's UI so the user does not have to wait for a server response for each and every key stroke. But as long as the frontend's validations are not stricter than those of the backend, the worst thing which will happen is that the user gets a later response about a piece of data which is malformed or inconsistent. Backend validation, however, should be the higher authority - it should contain all checks which are necessary to prevent the main business logic to malfunction. So when validation logic is maintained and extended, the focus for correctness should be on the backend, following the mantra "make things right, then make things fast - in that order".

Moreover, having the same logic in two places (even in different layers) is much more acceptable when the logic is not overly complicated and changes rarely, maybe only every few months or years, and not at run time. To make this more clear, let us look at your dynamic parameter example. In this validation and its implementation I spot mainly three different types of logic:

  1. The logic which data type each parameters has. For example, SERVERPATH shall be a parameter of the type "URL".

  2. The rules how the content of a text field has to be formed to represent a specific data type (like the format description of a valid URL, as it is written down in RFC 1738 - independent from a specific implementation).

  3. Finally,there is also the logic which connects the type formatting rules to a certain regex (which can be regex engine specific).

For (1), since the parameter definitions change dynamically at run time, there should definitely be a single source of truth. Having the frontend and the backend a different "opinion" about what parameter is a URL, or a date/time, or a an integer, has a high risk to lead to bugs. So a single parameter table which the frontend as well as the backend both use as a reference is a sensible approach.

For (2), the rules how certain data types have to look like when entered at a UI are relatively stable. As long as your frontend and your backend uses mainly standard data types like integers, strings of certain length, float values or calendar dates, it is perfectly acceptable to have two implementations for each of those type validations, one at the backend in its programming language, and one at the frontend in its - probably different - language. You won't introduce a maintenance headache this way, since all popular high level programming languages provide means for a simple implementation of such type checks. Even the rules for standard URLs are pretty stable (as long as you don't need to validate specifics like which top level domains or protocols are allowed, or whether the URLs are on some blacklist). This means, your idea of having a specific type field instead of a regex may require a duplicated type check implementation on frontend and backend. But that's an acceptable one, as long as you don't need a long, dynamic list of exotic data types.

(3), the idea of mapping every parameter directly to a regex (without having the abstraction of a type in between) is a half-baked approach to avoid similar code for the necessary type checks in (2). Unfortunately, this comes with its own hassles. As you have written by yourself, duplicating the same regex for the same data type over and over again is not the best idea. Though the rules how a general data type has to look like are mostly stable, having the rule how to map this to a regex a few dozen times in your system can introduce long-term maintenance issues. One can resolve this by using two tables: one type table listing the allowed types and their regexes, and a parameter table which refers to this type table. As Arseni has already explained, you may need not just one regex, but two different ones (one for the frontend and one for the backend).

But there are more problems. What if your application wants to support localized input and adapt the validations dynamically (for example, which decimal separator has to be used for a floating point number, or how dates are formatted)? A static regex is obviously not enough for this case, you may need either localized regexes or a dynamic regex creation, maybe with some template mechanic. What you typically want here is a frontend which behaves (and validates) localized, but converts data to a region-agnostic format before transferring it to the backend. This requires far more than a simple descriptive regex.

So in short:

  • Yes, using specific types instead of just one regex per parameter is definitely the more robust approach, and it will allow validations (and maybe conversion according to local settings) for which a static regex isn't enough. Using specific code for each of the types in frontend and backend produces some duplication, but this is often an acceptable form of duplication, at least for a small fixed number of types.

  • When there is a requirement to allow the users of your system introduce their own user-defined types, with the idea of regexes for validation of these types, I would recommend to make this an additional feature, to a list of predefined, maybe localizable data types. Still, you should check whether your frontend and backend can process the regexes by using the same underlying regex engine. Another approach could be to use a simplified pattern description, or a simplified common regex subset. Another approach could be to rely only on backend validation for these special types, given they occur seldom enough that its acceptable for the user for not getting immediate failure response during typing. As a last resort, you may let the user specify two different regexes, one for the frontend and one for the backend.

Improve this answer
10

Ok, suppose we got it settled that both the frontend and the backend must validate input.

If that is a settled decision, then it inherently implies validation code appearing in both layers. It can't be the same code because the two layers are written in different languages and run in different execution contexts. I guess the question is then how to minimize the maintenance burden and risk of skew associated with that. Some of the possibilities are:

  • Use some kind of parameterized or rule-based validation, where you have separate validators in the two layers consuming the same validation rules. This is relatively simple if you need to perform only static validations on one field at a time, but much more complex if you need to do cross-field validations or validations involving an underlying or external data source.

    Your idea of validating by matching against a regex would fall into this category, but as you seem to recognize, that has limited capabilities by itself.

  • Generate validation code for each layer from a single source of truth via a code generator. That source could be, but does not need to be, the relevant code for one of the layers.

    You might very well need to write the code generator. Or this is the kind of thing that an AI coding agent does very well, if you're willing to integrate that into your workflow.

  • Add a suitable validation API in the backend that shares code with the validations performed by the primary API entrypoints. The frontend would then affirmatively validate by using the validation API. It is of course possible to argue that the frontend is not actually doing validation in this case, and I would agree that it is not performing independent validation. But you still have code in the frontend that triggers validation and responds to validation results, so perhaps this satisfies the objective behind the rule (see also below).

Consider also whether the front end needs to perform all of the validations that the back end does. The primary purpose of front-end validation is to improve the user experience by providing more timely and better integrated validation feedback, whereas the purpose of back-end validation is to protect the application's data and core logic. The back end typically needs all the validations, but the front end might be able to get away with a subset. The best choice of logic-sharing approach may depend on what logic actually needs to be shared.

Improve this answer
2
  • 2
    +1 for the last paragraph, specifically. Often time the front-end can get away with only lightweight validation. Consider for example when a website asks for contact information (e-mail or phone number): the front-end only validates that the input looks like a valid email or phone number, and then the backend validates that the user actually "owns" the email or phone number by sending a code and asking the user to input it. Commented Jun 29 at 10:13
  • One additional alternative you could add: Create the validation logic as a shared library, which can be compiled for all platforms. With web-assembly you can run many backend-languages directly in the browser and could directly call the same methods which the backend uses for validation. Commented Jun 30 at 11:23
6

Ok, suppose we got it settled that both the frontend and the backend must validate input.

But how would you avoid duplication of validation logic across several application layers? The problem is you'd have to keep them in sync every time the logic changes.

Simple. Don't.

The point of validation logic isn't that input only has one canonical form. It's that every layer that has an opinion about that form needs a chance to express that opinion.

So the answer is don't keep them in sync. Give any layer that cares a chance to have their say.

The way you keep that from becoming a nightmare where every layer want's something different is to make sure most layers don't care about inputs form. For some layer's input can be nothing more than a pointer.

Do that and having several application layers isn't an issue. It's only having several input opinionated layers that's an issue.

Now I agree that it's good to have both the front end and the backend doing input validation. But the front end doesn't have to be in sync with the backend. It simply needs to be more restrictive than the backend. But that's only if you want to keep input errors to nice friendly red dots that prevent the user from submitting garbage. If you're OK with the user experiencing an error that came from the backend you don't even need them to be in sync.

If you're not ok with that, a sneaky trick is to log every time the backend sends out such an error. That log becomes a todo list of ways you could tighten down the front end validation.

But understand, even in that situation the front end is perfectly within it's rights to validate input based on it's own needs.

  • Just because the front end will tell you that you've misspelled a word doesn't mean the back end has to check for that
  • And just because the back end will insist that a user name be unique doesn't mean the front end needs a hash of every user name to enforce that

They don't need to be perfectly in sync. Simply compatible.

Improve this answer
8
  • 2
    "The frontend simply needs to be more restrictive than the backend." - em no, in most cases it is the opposite. The backend's validation is always higher authority, when the backend starts to rely on a strict validation of the frontend, you can be sure it will be bypassed sooner or later. Frontend validation is only a performance optimization, but when the frontend can already filter 90% of wrong user inputs directly, that helps to present users a faster feedback of what the backend should refuse to accept either. Commented Jun 29 at 8:55
  • 1
    ... and to your example: when a backend accepts a 100 character field (and can process this correctly), there is no point in a frontend which does accept only 50 characters in that field - that would be a nonsensical, artificial restriction. But when a backend can only process 100 characters correctly, it should validate this and refuse to accept 101 characters by its own validation, not relying on a front end's validation. Commented Jun 29 at 9:03
  • 4
    @DocBrown I never said the backend isn't the higher authority. I said if you make the frontend more restrictive than the backend you can avoid making the user see errors from the backend. The frontend validation is never a good excuse to not have backend validation if for no better reason than someone, sometime, could come up with a second frontend. Commented Jun 29 at 10:48
  • 7
    @candied_orange Your own argument of "someone, sometime, could come up with a second frontend" is precisely why your generalized advice of having a more restrictive frontend does not hold up. The backend should be an exact representation of the validation logic. The frontend should be there to catch unreasonable back and forth trips, but it might not always be able to cover for everything (e.g. an expensive validation check should not be run every time the user changes the value in a textbox, it would be an unreasonable hit to performance with insufficient ROI from validating early) Commented Jun 29 at 23:42
  • 1
    @candied_orange: if you want to avoid making the user see errors from the backend, you need to make the frontend equally strict as the backend, but not stricter. Commented Jun 30 at 7:57
3

Having layers doesn't mean that everything in the code base must be assigned to one layer and remain ignorant of other layers except their neighbours. Static validation routines that check formats, ranges etc. very much belong into utility/library/whatever-you-call it modules that should be visible everywhere.

Even if your front-end and back-end are written in different languages, it's relatively easy to auto-generate checking routines in one language from the patterns declared in another. Depending on how big and how important the validation task is in your project, doing this can be very worthwhile.

Improve this answer
1
  • 1
    Thank you, but it's too vague. What do you mean by "auto-generate checking routines in one language from the patterns declared in another"? I don't think you can reliably autogenerate anything if it's business logic. And if it's just format, you don't need to autogenerate at all Commented Jun 27 at 15:43
3

One approach, that we've used successfully in the past, is to have all the validation rules defined server-side and to then perform the validation via an API call.

In terms of configuration, there are FIELDS (e.g. User.EmailAddress) that have a TYPE (e.g. Email), which is backed by a class and, optionally, some CONFIGURATION (e.g. IsMandatory=true, MaxLength=255). Esoteric one-off fields still have their own single-use type class, which might extend another class to re-use/tweak the functionality.

Then there are FORMS, which are groups of FIELDS plus some additional validation rules (e.g. if User.Occupation == 'Other' then User.OccupationOther has IsMandatory set to true).

Whenever a front-end edit is made, the FormID plus the full set of form fields is sent back to the server API, which responds with an array of [FieldID => ErrorMessage] entries (which might be empty if there are no errors) and the client then updates the form to display these messages to the user in the appropriate location.

This method means there is only one implementation of the validation logic, on the server, but it is applied in all situations. In fact, we also use the same validation code to handle CSV bulk imports, data integrity tests, command-line scripts and a bunch of other things.

The other advantage of this method, is it allowed us to implement an auto-save feature incredibly cheaply!

The disadvantage, obviously, is a round-trip to the server to perform the validation, which increases server load and may affect perceived application performance. This part might need a bit of tweaking to get the best user experience; You don't want it to fire on every keystroke, but equally you don't necessarily want to wait until the control has lost focus, either, so a bit of fine-tuning was required to get this right.

Improve this answer
2

ASP.NET Core approach

One interesting solution that you can check and use as an inspiration is what is used by ASP.NET Core.

There, you define your model and use attributes on properties to specify which validation should be used for them. For instance, a string can end up with a [Required] and a [StringLength(50)] attributes, while an integer can have an attribute such as [Range(0, 10)].

Then, it's up to the framework to deal with both server side and client side validations—at least if you configured it to do so. No duplication on your side.

Outside the .NET world, the same approach can be reinvented (assuming there are no third party libraries that already to exactly that) by:

  1. Creating a bunch of validators that can be expressed both in a form of server side and client side code.

  2. Creating an interface model where individual elements can be decorated with those validators. How exactly this needs to be done depends on the language. .NET uses Reflection. One can also imagine a model being described in a form of a JSON that can be processed by virtually any popular programming language, would it be server side or client side.

  3. Deciding whether the model is shared with client side verbatim, and it's up to the client side to “own” its validation logic, that is, to know that [Range(0, 10)] has to be translated to a specific JavaScript code, or make JavaScript generation happen server side, leaving to the client just the task of executing it blindly.

Same language

Another possibility is to have the validation rules expressed in a single language that can either run both server side and in browser through WebAssembly (examples: Go, Rust), or be transpiled to JavaScript (examples: Kotlin, Dart).

Another possibility is to write the rules in JavaScript and let your server side app run JavaScript to do validation (or be fully written in JavaScript, if it's a Node.js application).

This could be an interesting approach compared to my earlier suggestion in a situation where you're constantly adding complex validation rules. This being said, a form with lots of complex validation rules is usually not sustainable anyway—and brings suffering for the end users.

One idea is to have a DB table type|regex

Caution with that.

The same regular expression won't necessarily execute in the same way on server side and in client side. An example of such difference is Unicode awareness for character classes:

JavaScript:

>> /^0-(\w+)[?!]$/.test('0-Hi!')
true
>> /^0-(\w+)[?!]$/.test('0-Έλα!')
false

Python:

>>> import re
>>> re.match(r'^0-(\w+)[?!]$', '0-Hi!') is not None
True
>>> re.match(r'^0-(\w+)[?!]$', '0-Έλα!') is not None
True

Assuming validation is made in server side anyway, security may not be your concern here. However, you are sure to have headaches when the same regular expression would match a string in some cases, but not match the same string in other cases.

Improve this answer
3
  • I'm looking into ASP .NET and only see cross-boundary validation propagation for Blazor. Is there a way to validate in JS or TS? Commented Jun 29 at 10:01
  • Not sure about your question, @Basilevs. Is this what you're looking for? Commented Jun 29 at 18:49
  • Thanks, looks similar, yes. Commented Jun 29 at 23:39
1

You are on the right track changing a per parameter regex to a parameter type, reducing the need to provide per parameter validation definitions.

However. I think you have got caught up on a problem that isn't a problem. I would challenge you to count the number of Parameter Types you need and think about the validation for each in different languages.

For Example .net has Uri.TryCreate, a javascript Date entry control will likely return a Date object. If you have say 10 types to validate, duplicating the logic per language used is not a problem and likely the best solution.

Sure you might have some custom types where you still use a regex, but even her you can have your backend write the hardcoded regex into the html/javascript generated for the page.

Improve this answer
-2

Could you list what has been considered to get to the conclusion

that both the frontend and the backend must validate input.

?

'cause frontend and backend validations are supported by different considerations. All the business validations are in the middleware, all data integrity validations are in the boundaries, frontend and backend (backend referring the persistence environment). Applied to the particularities of the question URL validation is business validation since the business decided to have URL addresses to point to external resources so the middleware, where all the business validations are implemented, must have knowledge about what are accepted values for each parameter. The frontend should have knowledge about what parameters are compulsory. To reduce bandwidth consumption it might be the frontend to have knowledge about some business requirements and validate data accordingly, though that's just optimisation and its trade-off(s) where for data intensive applications saving bandwidth prevails to logic duplication that same way avoid duplication is supported by DRY principle it is supported by write everything twice (WET) principle.

1
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. Commented Jun 30 at 16:24

Your Answer

Draft saved
Draft discarded

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.