# API Workflow Guide How to add or change an API endpoint in this project, end to end. Read `docs/PROJECT_ARCHITECTURE.md` first; read `AGENTS.md` before every task. > **This is a base codebase.** The technical steps below are fixed and reused verbatim. > The *domain* they operate on is new: most business models do not exist yet. When a task > needs a model that is not there, design it first via `docs/DOMAIN_MODELING_GUIDE.md` — > do not bend the feature to fit an unrelated old model. > > Two task shapes, one workflow: > > | Shape | Extra step | > |---|---| > | **Existing new-domain module** — the model already exists | none — start at § 2 | > | **New business module** — the model does not exist yet | **§ 2b Domain Design** before writing any file | --- ## 1. When to use a Full Scan A full project scan happens **once per codebase**, not per task. Its output is `docs/PROJECT_ARCHITECTURE.md`. Re-run a full scan only when: - The architecture doc is missing or contradicted by the code. - A large structural change landed (new mount, new layer, Express/Mongoose major upgrade). - You are onboarding into a part of the system the doc does not cover at all. Never full-scan because a task "feels big". `adminRoute.js` (~1.7k lines) and `returnObject.js` (~5.9k lines) make blind re-scanning expensive and low-signal. --- ## 2. When to use Targeted Discovery **Every task after the initial scan.** Targeted Discovery means: open only the files in the vertical slice you are about to change, plus the shared contracts they touch. The standard slice for an API task: ``` src/routes/api/Route/Route.js ← route registration src/routes/api/indexRoute/indexRoute.js ← is the bucket wired? src/utils/validations/api/.js ← input rules src/controllers/api/Controller.js ← thin delegate src/helpers/api/.js ← business logic src/models/Model.js ← schema src/helpers/returnObject/returnObject.js ← ONLY the relevant exports src/helpers/enums/.enum.js ← statuses src/locales/ar/.json + src/locales/en/.json ← messages ``` Plus, once per session (they rarely change): `ApiResponse.js`, `ApiError.js`, `error.handler.js`, `showErrorsApi.js`, `sharedController.js` (`checkValidations`, `uploadAnyFile`). Use `grep`/search to locate exact symbols instead of reading whole large files. For `returnObject.js`, jump to the specific `exports.` function. **If a file in that slice does not exist yet, that is expected** — this is a base codebase and the new domain is being built. A missing `src/models/Model.js` means "design it", not "find a substitute". --- ## 2b. Domain Design (new business modules only) Run this **before** Targeted Discovery of implementation files, and before writing a single schema line. Full checklist: `docs/DOMAIN_MODELING_GUIDE.md`. 1. **Confirm it is genuinely new.** ```bash ls src/models | grep -i ``` If a same-named model exists, determine whether it is a **new-domain** model (reuse it) or an **old-domain** model (do not reuse it — raise the naming conflict with the human). 2. **Read the analysis** for the entity. No analysis → ask; never invent the domain. 3. **Produce the design**: fields · relationships · status enum · required/optional · indexes · sensitive fields · file/image fields. 4. **Get it reviewed** before implementation. Reuse from the base: schema class style, `{ timestamps: true, versionKey: false }`, frozen enums, `mongoose-sequence`, `{ i18n: true }`. Do **not** inherit the base's modeling debts: missing indexes, mixed soft-delete conventions, zero `select: false`, and `pre(/^find/)` auto-populate. --- ## 3. Steps to implement any API 1. **Read the rules** — `AGENTS.md`, and the architecture doc if this is your first task. 2. **Targeted Discovery** — open the slice above; note the exact patterns you will copy. If the module is new, do **§ 2b Domain Design** first. 3. **Publish the Discovery Report** (7 sections, see `AGENTS.md`) before writing code. For a new module, include the Domain Design in it. 4. **Model** — new module: create the schema per `docs/DOMAIN_MODELING_GUIDE.md` (declared indexes, enum-backed status, `select: false` on sensitive fields, no `pre(/^find/)` auto-populate). Existing new-domain module: add/adjust fields only if genuinely required. Reuse enums from `src/helpers/enums/`. Never add a field you will not validate, and never extend an old-domain model to host new behaviour. 5. **Validation** — add a `validateX()` static method returning an `express-validator` chain array. Every field you intend to consume must be declared, or `matchedData` will drop it. 6. **Route** — register in the correct class and the correct bucket (`unRequireAuthRoutes()` for public, `registerRoutes()` for authenticated), wrapped in `Validator.getValidationChain(Validator.validateX())`. 7. **Controller** — one-line delegate in `src/controllers/api/Controller.js`. 8. **Business logic** — implement in `src/helpers/api/.js` following the canonical `try { checkValidations → work → ApiResponse } catch { errorHandler }` body. 9. **DTO** — add or extend the serializer in `@returnObj`. Never return the document. 10. **i18n** — add every message key to both `src/locales/ar/` and `src/locales/en/`. 11. **Docs** — update the API contract for the touched endpoints only (see `docs/SWAGGER_GUIDE.md`). 12. **Test** — extract any pure rule into a helper and add a `test/*.test.js` runner; `npm test` runs that explicit glob automatically. Keep DB/server/external integration tests out of the default unit suite, plus perform manual QA against `npm run start:dev`. 13. **Final Report** — 10 sections, see `AGENTS.md`. --- ## 4. Route responsibility The route layer decides **path, method, auth bucket, and which validator runs**. Nothing else. ```js // src/routes/api/Route/Route.js class ProductRoutes { constructor() { this.router = express.Router(); } unRequireAuthRoutes() { // public — mounted BEFORE requireAuth const router = express.Router(); router.get('/productDetails', ProductValidator.getValidationChain(ProductValidator.validateDetails()), product.details); return router; } registerRoutes() { // authenticated — mounted AFTER requireAuth this.router.post('/product', ProductValidator.getValidationChain(ProductValidator.validateCreate()), product.create); return this.getRouter(); } getRouter() { return this.router; } } module.exports = new ProductRoutes(); ``` Rules: - No business logic, no DB calls, no `res.*` in a route file. - A new domain must also be required and invoked in `src/routes/api/indexRoute/indexRoute.js`, in the right position relative to `requireAuth`. - Do not add per-route auth middleware; the bucket is the auth decision. - Be aware that `requireAuth` whitelists a handful of paths (`/home`, `/productDetails`, `/client`, …) so those tolerate anonymous requests even in the authenticated bucket — handle a possibly-absent `req.user` there. --- ## 5. Validation responsibility The validation layer owns **shape, type, range, format, existence, and uniqueness** of input. ```js class ProductValidator { static getValidationChain(fn) { return [fn, showErrors]; } static validateCreate() { return [ body('title').trim().notEmpty().withMessage(createRequiredMessage('title')), body('price').isFloat({ min: 0 }).withMessage(/* i18n */), body('departmentId').isMongoId().bail() .custom(async (value, { req }) => { const dept = await Department.findOne({ _id: value, status: StatusEnum.ACTIVE }); if (!dept) throw new Error(/* i18n message object */); req.department = dept; // hand the loaded doc to the controller }), ]; } } ``` Rules: - Declare **every** field you will read. Undeclared fields never reach the controller. - Use `libphonenumber-js` + `persianjs` for phone normalization, as `auth.js` does. - Validate uploads with `GlobalValidator.validateImageFile` (magic bytes), not client MIME. - Put existence/ownership lookups here when the result is also needed downstream; attach it to `req` rather than re-querying in the business layer. - Never put a password, OTP, or token into a validation error message. --- ## 6. Controller responsibility Thin delegate. That is the whole job. ```js // src/controllers/api/productController.js const Product = require("@root/helpers/api/Product"); module.exports = { create(req, res) { Product.create(req, res); }, details(req, res) { Product.details(req, res); }, }; ``` If you find yourself writing an `if` in an API controller, it belongs in `helpers/api/`. --- ## 7. Service responsibility (`src/helpers/api/.js`) Owns business rules, persistence, side effects, and the response. ```js async create(req, res) { const { lang } = req.headers; try { const dataBody = checkValidations(req); // whitelisted input const id = initId(); const dir = makeDir(`users/providers/${PRODUCT}/${req.user._id}`); if (req.files?.image) dataBody.image = await uploadAnyFile(req, "image", dir, "image"); const product = await Product.create({ ...dataBody, _id: id, provider: req.user._id }); await sendNotification({ sender: req.user, receiver: admin, translatedMessage, data }); return res.send(new ApiResponse( 'success', i18n.__({ phrase: 'product.createdSuccessfully', locale: lang }), StatusCodes.OK, await returnObject.product(product, lang), )); } catch (error) { console.log("Error:", error); throw errorHandler({ res, statusName: 'exception', i18nMessage: 'common.returnDeveloper', lang }); } } ``` Rules: - Always `const { lang } = req.headers;` at the top; pass `lang` into every i18n call and every `errorHandler`. - Business failures use a specific key: `errorHandler({ res, statusName: 'fail'|'notFound'|'unauthorized'|'blocked', i18nMessage, lang })`. Reserve `'exception'` for the outer catch. - `errorHandler` responds **and then throws** — that is intentional; keep `throw errorHandler(...)`. - Never `console.log` the request body on auth-adjacent flows (it can contain a password). - External calls (SMS, push, OTO, Agora) go through `src/services/*`, never inline `axios`. --- ## 8. Model responsibility Schema, defaults, indexes, hooks. No request awareness. - File: `src/models/Model.js`, flat directory. - Style: `class XSchema extends Schema` (or `extends UserSchema` for user-like entities), instantiated once, exported via `model('X', schema)`. - `{ timestamps: true, versionKey: false }` — always. - Statuses come from `src/helpers/enums/*.enum.js`. - Bilingual text fields use `{ type: String, i18n: true }` + `mongoose-i18n-localize`. - Never store a plaintext secret. Password hashing already exists in the base `pre('save')`. **For new-domain models, additionally (see `docs/DOMAIN_MODELING_GUIDE.md`):** - Declare every index the endpoints actually query on — the base declares indexes in only 8 of 67 models; do not inherit that. - Apply the new domain's single soft-delete convention consistently (`status` enum + `deletedAt`), with partial unique indexes where needed. - `select: false` on anything sensitive. The base relies on the DTO alone (0/67 use `select: false`); new models use both layers. - **No `pre(/^find/)` auto-populate.** 31 of 67 base models do this and it fires on every query including counts. Populate explicitly in the service layer. > ⚠️ Editing a **base** model (`userModel.js`, `providerModel.js`, …) ripples widely — > `userModel.js` alone affects `Client`, `Provider`, and `ProviderMeta`. Prefer creating a > new-domain model over widening an old one. --- ## 9. Safe response Every API response body is built by `@returnObj` + `ApiResponse`. ```js new ApiResponse(key, message, status, data) // → { key, message, status, data } // with type === 'api' → also { paginate: { currentPage, lastPage, perPage, total } } ``` Never include in `data`: - `password`, `activationCode` / OTP, `activationCodeExpire` - raw JWTs other than the single `token` field on sign-in/activate - internal `__v`, Mongo `$`-operators, raw error objects, stack traces - any `.env` value Always include: - `id` (mapped from `_id`) - absolute file URLs built with `returnObject.filePath` / `productImagePath`, never raw filenames - localized `*Text` fields via `i18n.__({ phrase, locale: lang })` - dates formatted with `moment(...).format('YYYY/MM/DD')` to match existing DTOs > Known deviation: `returnObject.client` currently returns `activationCode`. Do not copy > that into new DTOs. --- ## 10. Error handling | Situation | What to do | |---|---| | Input invalid | Let the validator + `showErrorsApi` handle it. Do not re-check in the service. | | Business rule violated | `throw errorHandler({ res, statusName: 'fail', i18nMessage: 'x.y', lang })` | | Entity missing | `statusName: 'notFound'` | | Not the owner / no token | `statusName: 'unauthorized'` | | Account blocked | `statusName: 'blocked'` | | Unexpected | outer catch → `statusName: 'exception'`, `i18nMessage: 'common.returnDeveloper'` | Key → HTTP: `success` 200 · `needActive` 203 · `fail` 400 · `notFound` 400 · `unauthorized` 419 · `blocked` 423 · `exception` 500. For compatibility, `responseError('notFound', ...)` currently emits `key: "fail"` with status 400; the contract tests lock this behavior until a separately approved client migration. Never `res.status(500).json(error)`. Never surface `error.message` from a driver or third-party SDK to the client. --- ## 11. Security checklist Run through this before declaring an API task done: - [ ] Every consumed field is declared in a validator (no `req.body.x` for an undeclared `x`). - [ ] Ownership is verified — a provider can only touch their own products/auctions; a client only their own cart/orders. `req.user._id` is in the query, not just the body. - [ ] No `password`, `activationCode`, token, or `.env` value in the response. - [ ] No `console.log` of request data. Use the structured logger with a minimal context; never pass bodies, query strings, headers, cookies, credentials, OTPs, or secrets. - [ ] Preserve the response `X-Request-Id` when reporting failures; do not add it to the response envelope or accept arbitrary request IDs. - [ ] File uploads validated by signature (`validateImageFile`), stored under `public/assets/uploads/...` via `makeDir` + `uploadAnyFile` — no client-controlled path segments. - [ ] IDs validated with `isMongoId()` before any query. - [ ] Money/balance changes go through `helpers/wallet/checkBalance` and write a `BalanceHistory` / `FinancialTransaction` row. - [ ] Status transitions use enums, not string literals, and reject invalid source states. - [ ] Public endpoints (`unRequireAuthRoutes`) leak nothing that requires auth. - [ ] Dashboard route (if touched): consult `docs/CSRF_AUDIT.md`; `csrfProtection` present and `uploadsFiles()` before it on multipart. - [ ] Static assets use an explicit public/build directory. Never mount the repository root. - [ ] No new package added. - [ ] New model (if any): sensitive fields marked `select: false`, DTO omits them, and no old-domain model was extended to host new behaviour. --- ## 12. Testing checklist - [ ] Pure rules extracted into `src/utils/` or `src/helpers/` and covered by a `test/.test.js` runner (plain Node + `assert`). - [ ] `npm test` run, real output reported — including failures. - [ ] Tests needing MongoDB, a listening server, or an external service use a separate, explicitly named integration script and are not forced into default `npm test`. - [ ] Manual QA against `npm run start:dev`: - [ ] happy path - [ ] each validation failure returns the right `key`/`status`/localized `message` - [ ] `lang: ar` and `lang: en` headers both produce correct messages - [ ] unauthenticated call to an authenticated route → `key: "unauthorized"`, 419 - [ ] wrong-owner call → rejected - [ ] pagination (`page`, `limit`) if the endpoint lists - [ ] Postman/OpenAPI entry updated for the touched endpoints. ### Concrete verification commands ```bash npm run start:dev # HTTP on LOCALE_ADDRESS:HTTP — no TLS certs required ``` ```bash # public endpoint — note the required `lang` header curl -i -X POST "http://localhost:$HTTP/api/signup-client" \ -H "lang: ar" \ -F "name=اختبار" -F "countryCode=+966" -F "phone=0512345678" \ -F "password=Passw0rd!" -F "confirmPassword=Passw0rd!" # authenticated endpoint curl -i "http://localhost:$HTTP/api/profile" \ -H "lang: en" -H "Authorization: Bearer $TOKEN" # missing token → expect key "unauthorized", HTTP 419 curl -i "http://localhost:$HTTP/api/profile" -H "lang: ar" # missing/invalid lang header → rejected by GlobalValidator.validateLang() curl -i "http://localhost:$HTTP/api/profile" -H "lang: fr" -H "Authorization: Bearer $TOKEN" ``` Assert on the `key` field, not only the HTTP status — mobile clients branch on `key`, and `needActive` is returned with HTTP 200. Never paste a real token, password, or OTP into a report, a test fixture, or a commit. --- ## 13. Final report checklist 1. Files inspected 2. Files created 3. Files modified 4. Endpoints added/changed (method + path + auth bucket) 5. Validation rules (field → rule → error key) 6. Documentation updates 7. Security checks performed 8. Manual QA performed 9. Automated test results (verbatim) 10. Not implemented / deferred, and why --- ## 14. Variant — Dashboard task workflow Dashboard tasks use the same discovery discipline but a different slice and a different implementation standard (see `AGENTS.md` → *Dashboard Implementation Standard*). ### Slice to inspect ``` src/routes/dashboard/adminRoute/adminRoute.js ← grep the section, don't read all 1676 lines src/utils/validations/dashboard/.js src/controllers/dashboard/Controller/Controller.js views/admin//{index,create,edit,show,dataTable,tds}.ejs views/admin/includes/ ← reusable partials src/helpers/pagination/pagination.js src/helpers/reports/ ← the audit `report(...)` helper src/locales/{ar,en}/.json ``` ### Steps 1. **Grep, don't read.** `grep -n "// Clients" -A 40 src/routes/dashboard/adminRoute/adminRoute.js` to find the section for your module. 2. Copy the chain of a neighbouring route in the same section verbatim — the middleware order is load-bearing (`uploadsFiles()` before `csrfProtection` on multipart; `/x/filter` before any `/x` regex). 3. Controller: GET renders EJS with the full context object (`title`, `currentMenu`, `user: req.admin`, `csrfToken`, `lang`, `i18n`, `moment`); mutations respond with `ApiResponse` JSON carrying `data.url` for the client-side redirect. 4. Views: reuse the 6-file CRUD set and the shared partials in `views/admin/includes/`. Do not invent a new page skeleton. 5. Audit every mutation: `report(req.admin._id, { ar, en, ur }, req.method, req.url)`. 6. If you added or renamed routes, run `npm run listPermissions` and include the regenerated `src/helpers/permissions/permissions.js` + `permissions.json` diffs in the report. 7. Manual QA: log in at `/dashboard/login`, exercise list → filter (AJAX) → create → edit → delete, and confirm the CSRF token round-trips on multipart forms. > ⚠️ Dashboard RBAC is currently a no-op (`authorization.js` deny branches are commented > out). Adding a permission string to a role changes the checkbox UI but grants nothing > and blocks nothing. Do not describe a dashboard task as "permission-gated" unless you > were explicitly asked to re-enable enforcement. The reviewed Phase 1 policy boundary > and staged rollout are documented in `docs/RBAC_DESIGN.md`. --- ## 15. Variant — Socket & Cron tasks **Socket:** add the rule to `listeners/socketManger/Validation/index.js`, the handler to `Helper/index.js`, and wire it in `socket.js`. Emit with the same `ApiResponse` envelope. QA with `node scripts/test-call-socket-flow.js` as a reference driver. **Cron:** export the handler from `src/helpers/cronJopFn/cronJobFn.js` **and** schedule it via `addCronJobToDB`. Both halves are required — a `fn` name in the DB with no exported handler is silently marked FINISHED at boot. Handlers must be idempotent, because `scheduleCronJobs()` fires past-due deadlines immediately on restart. --- ## Worked example — Client Signup + Provider Signup > This is a walkthrough of how the workflow applies. The endpoints below **already exist in > the base**; the example shows the process, not a change to make. > > **Base vs new for this task.** Auth is a *hybrid*: the mechanism is base, the domain is new. > > | Part | Classification | Action | > |---|---|---| > | JWT issuance (`utils/token/token.js`), `UserToken`, `passport` strategy, `requireAuth` | **Base — technical spine** | Reuse as-is | > | OTP generation, activation flow, device registration | **Base — mechanism** | Reuse the pattern | > | Validation chain style, response envelope, error keys | **Base** | Reuse as-is | > | `Client` / `Provider` **schemas and fields** | **New domain** | Re-derive from the analysis — do not assume the base's `userType: [store, haraj]`, `approvalStatus`, `commercialNumber`, or `ProviderMeta` mirror belong to the new product | > > So step 2 below inspects the base to learn *how* signup works, while step 4 onward > re-derives *what* a Client and a Provider are. If the new analysis differs from the base > schema, the analysis wins and the model is redesigned per `docs/DOMAIN_MODELING_GUIDE.md`. ### 1. Read architecture docs `AGENTS.md` → `docs/PROJECT_ARCHITECTURE.md` (Auth section, Response Format, Error Handling). ### 2. Targeted discovery — auth slice only ``` src/routes/api/AuthRoute/AuthRoute.js src/routes/api/indexRoute/indexRoute.js src/utils/validations/api/auth.js src/utils/validations/api/global.js src/controllers/api/authController.js src/helpers/api/Auth.js src/models/userModel.js clientModel.js providerModel.js providerMetaModel.js src/utils/token/token.js src/utils/generateCode/generateCode.js src/services/sendSMS/sendSMS.js src/services/passport/passport.js src/helpers/returnObject/returnObject.js → exports.client, exports.provider src/locales/ar/auth.json src/locales/en/auth.json ``` Patterns found: - Both signup routes live in `unRequireAuthRoutes()` (public, before `requireAuth`). - Both go route → thin controller → `Auth.signupClient` / `Auth.signupProvider`. - Provider signup writes **both** `Provider` and `ProviderMeta` and copies the upload folder from `users/providersMeta/` to `users/providers/`. - Both return `key: "needActive"` at HTTP 200 — not `"success"`. - SMS dispatch is present but **commented out** in `Auth.js`. ### 3. Identify endpoints | Method | Path | Auth | Purpose | |---|---|---|---| | POST | `/api/signup-client` | public | Create a client, `active:false`, issue OTP | | POST | `/api/signup-provider` | public | Create provider + providerMeta, notify super-admin | | PATCH | `/api/activate` | public | Consume the OTP, activate, register device, issue JWT | ### 4. Validation Reuse `AuthValidator.signup()` (name / countryCode / phone / password / confirmPassword) and extend per user type — provider adds `userType ∈ [store, haraj]`, conditional `commercialNumber`, the all-or-nothing bank group, and `city`/`address`/`longitude`/`latitude`. Phone uniqueness is checked across `Client → Provider → ProviderMeta`, including the `oppositePhone` variant, excluding `status: delete`. ### 5. Controller ```js signupClient(req, res) { Auth.signupClient(req, res); } signupProvider(req, res) { Auth.signupProvider(req, res); } ``` ### 6. Service `Auth.signupClient` / `Auth.signupProvider` in `src/helpers/api/Auth.js`: `checkValidations` → `initId()` → `makeDir` → optional `uploadAnyFile` for `avatar` → `Model.create({ ...dataBody, _id, activationCode: await generateCode(), activationCodeExpire: Date.now() + 60_000 })` → (provider) `sendNotification` to the super-admin → `new ApiResponse('needActive', i18n.__({ phrase: 'auth.accountCreatedSuccessfully', locale: lang }), 200, await returnObject.client|provider(user, lang))`. ### 7. Update API docs Document only `POST /signup-client`, `POST /signup-provider`, `PATCH /activate`. Request bodies are `multipart/form-data` (avatar upload). Responses must **not** show `password` or `activationCode`. ### 8. Test - Unit: phone normalization + password-strength regex extracted and asserted. - Manual: duplicate phone → `key: "fail"`; weak password → `key: "fail"`; `store` without `commercialNumber` → `fail`; partial bank group → `fail`; success → `key: "needActive"`, HTTP 200, no OTP in the body. - Verify `Provider` **and** `ProviderMeta` rows both exist and the avatar folder was copied. ### 9. Report Emit the 10-section Final Report, including the fact that SMS delivery is commented out and therefore the OTP is not actually sent in the current code path.