Context Engineering

Which information should reach a model, in what form, and within what budget, for a given task?

Models do not know a company’s data, and giving them everything is neither possible nor safe. Too little context produces invented answers; too much buries the relevant fact.

Selecting context is a ranking and budgeting problem: score candidate sources by relevance, then choose a subset that fits the budget, keeping the source of every passage.

Relevance by cosine similarity between embeddings E, then selection of passages with value v and size t under a budget B.

1. Index sources together with their provenance.

2. Retrieve candidates by similarity and filter by permissions.

3. Select a subset under the budget.

4. Return the answer together with the passages it used.

1. Question

2. Retrieval

3. Permission filter

4. Budgeted selection

5. Model

6. Answer with sources

No published experiments. The publication on cosine similarity derives the relevance score with a worked example.

Enterprise Knowledge Graph

No original research results are claimed. This area is presented as a technical perspective with a computable demonstration on synthetic data.

• Similarity measures resemblance, not truth.
• Embedding quality sets a ceiling on retrieval.
• Budgets trade completeness for cost.

Background reading by other authors. These are not Menlo Tech publications, and no Menlo Tech paper is cited because none has been published.

• Manning, C., Raghavan, P. & Schütze, H. Introduction to Information Retrieval. Cambridge University Press, 2008.
• Lewis, P. et al. "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks." NeurIPS, 2020.