LLM/khoj
1
0
Fork 0
mirror of https://github.com/khoj-ai/khoj.git synced 2026-05-13 21:41:41 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: LLM:1.26.2
Branches Tags
LLM:master LLM:release/1.x LLM:v2/add-live-voice-chat LLM:debanjum/miscellaneous-v2-changes LLM:use-user-specified-chat-model-for-intermediate-steps LLM:pr/hjamet/1109 LLM:create-data-tracer LLM:evals/tmp-distill-trains-of-thought LLM:load-web-app-from-desktop-app LLM:use-structured-index-to-improve-kb-retrieval LLM:speed-up-khoj-docker-builds LLM:features/support-sqlite-as-db LLM:regenerate-edit-messages-on-web-app LLM:improve-agents-pane-on-home-screen LLM:use-turnstyle-ux-for-khoj-chat LLM:improve-doc-chunking-and-support-timely-embedings LLM:speed-up-embedding-model-app-load-when-not-connected-to-internet LLM:panchayat
LLM:2.0.0-beta.28 LLM:2.0.0-beta.27 LLM:2.0.0-beta.26 LLM:2.0.0-beta.25 LLM:2.0.0-beta.24 LLM:2.0.0-beta.23 LLM:2.0.0-beta.22 LLM:2.0.0-beta.21 LLM:2.0.0-beta.20 LLM:2.0.0-beta.19 LLM:2.0.0-beta.18 LLM:2.0.0-beta.17 LLM:2.0.0-beta.16 LLM:2.0.0-beta.15 LLM:2.0.0-beta.14 LLM:2.0.0-beta.13 LLM:2.0.0-beta.12 LLM:2.0.0-beta.11 LLM:2.0.0-beta.10 LLM:2.0.0-beta.9 LLM:2.0.0-beta.8 LLM:1.42.10 LLM:2.0.0-beta.7 LLM:2.0.0-beta.6 LLM:2.0.0-beta.5 LLM:2.0.0-beta.4 LLM:2.0.0-beta.3 LLM:2.0.0-beta.2 LLM:2.0.0-beta.1 LLM:1.42.9 LLM:1.42.8 LLM:1.42.7 LLM:1.42.6 LLM:1.42.5 LLM:1.42.4 LLM:1.42.3 LLM:1.42.2 LLM:1.42.1 LLM:1.42.0 LLM:1.41.0 LLM:1.40.0 LLM:1.39.0 LLM:1.38.0 LLM:1.37.2 LLM:1.37.1 LLM:1.37.0 LLM:1.36.6 LLM:1.36.5 LLM:1.36.4 LLM:1.36.3 LLM:1.36.2 LLM:1.36.1 LLM:1.36.0 LLM:1.35.3 LLM:1.35.2 LLM:1.35.1 LLM:1.35.0 LLM:1.34.0 LLM:1.33.2 LLM:1.33.1 LLM:1.33.0 LLM:1.32.2 LLM:1.32.1 LLM:1.32.0 LLM:1.31.0 LLM:1.30.10 LLM:1.30.9 LLM:1.30.8 LLM:1.30.7 LLM:1.30.6 LLM:1.30.5 LLM:1.30.4 LLM:1.30.3 LLM:1.30.2 LLM:1.30.1 LLM:1.30.0 LLM:1.29.1 LLM:1.29.0 LLM:1.28.3 LLM:1.28.2 LLM:1.28.1 LLM:1.28.0 LLM:1.27.1 LLM:1.27.0 LLM:1.26.4 LLM:1.26.3 LLM:1.26.2 LLM:1.26.1 LLM:1.26.0 LLM:1.25.0 LLM:1.24.1 LLM:1.24.0 LLM:1.23.3 LLM:1.23.2 LLM:1.23.1 LLM:1.23.0 LLM:1.22.3 LLM:1.22.2 LLM:1.22.1 LLM:1.22.0 LLM:1.21.6 LLM:1.21.5 LLM:1.21.4 LLM:1.21.3 LLM:1.21.2 LLM:1.21.1 LLM:1.21.0 LLM:1.20.4 LLM:1.20.3 LLM:1.20.2 LLM:1.20.1 LLM:1.20.0 LLM:1.17.0 LLM:1.16.0 LLM:1.15.0 LLM:1.14.0 LLM:1.13.0 LLM:1.12.1 LLM:1.12.0 LLM:1.11.2 LLM:1.11.1 LLM:1.11.0 LLM:1.10.2 LLM:1.10.1 LLM:1.10.0 LLM:1.9.0 LLM:1.8.0 LLM:1.7.0 LLM:1.6.2 LLM:1.6.1 LLM:1.6.0 LLM:1.5.1 LLM:1.5.0 LLM:1.4.0 LLM:1.3.0 LLM:1.2.1 LLM:1.2.0 LLM:1.1.0 LLM:1.0.1 LLM:1.0.0 LLM:0.14.0 LLM:0.13.0 LLM:0.12.3 LLM:0.12.2 LLM:0.12.1 LLM:rm LLM:0.12.0 LLM:0.11.4 LLM:0.11.3 LLM:0.11.2 LLM:0.11.1 LLM:0.11.0 LLM:0.10.1 LLM:0.10.0 LLM:0.9.0 LLM:0.8.2 LLM:0.8.1 LLM:0.8.0 LLM:0.7.1 LLM:0.7.0 LLM:0.6.2 LLM:0.6.1 LLM:0.6.0 LLM:0.5.0 LLM:0.4.0 LLM:0.3.0 LLM:0.2.6 LLM:0.2.5 LLM:0.2.1 LLM:v0.2.0 LLM:v0.1.9 LLM:v0.1.8 LLM:v0.1.7 LLM:v0.1.6 LLM:v0.1.5-alpha
...
compare: LLM:master
Branches Tags
LLM:master LLM:release/1.x LLM:v2/add-live-voice-chat LLM:debanjum/miscellaneous-v2-changes LLM:use-user-specified-chat-model-for-intermediate-steps LLM:pr/hjamet/1109 LLM:create-data-tracer LLM:evals/tmp-distill-trains-of-thought LLM:load-web-app-from-desktop-app LLM:use-structured-index-to-improve-kb-retrieval LLM:speed-up-khoj-docker-builds LLM:features/support-sqlite-as-db LLM:regenerate-edit-messages-on-web-app LLM:improve-agents-pane-on-home-screen LLM:use-turnstyle-ux-for-khoj-chat LLM:improve-doc-chunking-and-support-timely-embedings LLM:speed-up-embedding-model-app-load-when-not-connected-to-internet LLM:panchayat
LLM:2.0.0-beta.28 LLM:2.0.0-beta.27 LLM:2.0.0-beta.26 LLM:2.0.0-beta.25 LLM:2.0.0-beta.24 LLM:2.0.0-beta.23 LLM:2.0.0-beta.22 LLM:2.0.0-beta.21 LLM:2.0.0-beta.20 LLM:2.0.0-beta.19 LLM:2.0.0-beta.18 LLM:2.0.0-beta.17 LLM:2.0.0-beta.16 LLM:2.0.0-beta.15 LLM:2.0.0-beta.14 LLM:2.0.0-beta.13 LLM:2.0.0-beta.12 LLM:2.0.0-beta.11 LLM:2.0.0-beta.10 LLM:2.0.0-beta.9 LLM:2.0.0-beta.8 LLM:1.42.10 LLM:2.0.0-beta.7 LLM:2.0.0-beta.6 LLM:2.0.0-beta.5 LLM:2.0.0-beta.4 LLM:2.0.0-beta.3 LLM:2.0.0-beta.2 LLM:2.0.0-beta.1 LLM:1.42.9 LLM:1.42.8 LLM:1.42.7 LLM:1.42.6 LLM:1.42.5 LLM:1.42.4 LLM:1.42.3 LLM:1.42.2 LLM:1.42.1 LLM:1.42.0 LLM:1.41.0 LLM:1.40.0 LLM:1.39.0 LLM:1.38.0 LLM:1.37.2 LLM:1.37.1 LLM:1.37.0 LLM:1.36.6 LLM:1.36.5 LLM:1.36.4 LLM:1.36.3 LLM:1.36.2 LLM:1.36.1 LLM:1.36.0 LLM:1.35.3 LLM:1.35.2 LLM:1.35.1 LLM:1.35.0 LLM:1.34.0 LLM:1.33.2 LLM:1.33.1 LLM:1.33.0 LLM:1.32.2 LLM:1.32.1 LLM:1.32.0 LLM:1.31.0 LLM:1.30.10 LLM:1.30.9 LLM:1.30.8 LLM:1.30.7 LLM:1.30.6 LLM:1.30.5 LLM:1.30.4 LLM:1.30.3 LLM:1.30.2 LLM:1.30.1 LLM:1.30.0 LLM:1.29.1 LLM:1.29.0 LLM:1.28.3 LLM:1.28.2 LLM:1.28.1 LLM:1.28.0 LLM:1.27.1 LLM:1.27.0 LLM:1.26.4 LLM:1.26.3 LLM:1.26.2 LLM:1.26.1 LLM:1.26.0 LLM:1.25.0 LLM:1.24.1 LLM:1.24.0 LLM:1.23.3 LLM:1.23.2 LLM:1.23.1 LLM:1.23.0 LLM:1.22.3 LLM:1.22.2 LLM:1.22.1 LLM:1.22.0 LLM:1.21.6 LLM:1.21.5 LLM:1.21.4 LLM:1.21.3 LLM:1.21.2 LLM:1.21.1 LLM:1.21.0 LLM:1.20.4 LLM:1.20.3 LLM:1.20.2 LLM:1.20.1 LLM:1.20.0 LLM:1.17.0 LLM:1.16.0 LLM:1.15.0 LLM:1.14.0 LLM:1.13.0 LLM:1.12.1 LLM:1.12.0 LLM:1.11.2 LLM:1.11.1 LLM:1.11.0 LLM:1.10.2 LLM:1.10.1 LLM:1.10.0 LLM:1.9.0 LLM:1.8.0 LLM:1.7.0 LLM:1.6.2 LLM:1.6.1 LLM:1.6.0 LLM:1.5.1 LLM:1.5.0 LLM:1.4.0 LLM:1.3.0 LLM:1.2.1 LLM:1.2.0 LLM:1.1.0 LLM:1.0.1 LLM:1.0.0 LLM:0.14.0 LLM:0.13.0 LLM:0.12.3 LLM:0.12.2 LLM:0.12.1 LLM:rm LLM:0.12.0 LLM:0.11.4 LLM:0.11.3 LLM:0.11.2 LLM:0.11.1 LLM:0.11.0 LLM:0.10.1 LLM:0.10.0 LLM:0.9.0 LLM:0.8.2 LLM:0.8.1 LLM:0.8.0 LLM:0.7.1 LLM:0.7.0 LLM:0.6.2 LLM:0.6.1 LLM:0.6.0 LLM:0.5.0 LLM:0.4.0 LLM:0.3.0 LLM:0.2.6 LLM:0.2.5 LLM:0.2.1 LLM:v0.2.0 LLM:v0.1.9 LLM:v0.1.8 LLM:v0.1.7 LLM:v0.1.6 LLM:v0.1.5-alpha

1621 Commits
1.26.2 ... master

Author SHA1 Message Date
Debanjum
9258f57dce Release Khoj version 2.0.0-beta.28 2026-03-26 09:03:47 +05:30
Debanjum
171ac5d243 Add deprecation banner to top of web landing page as well 2026-03-26 08:59:05 +05:30
Debanjum
fdd5fd8f74 Fix getting billing config to show deprecation banner on Khoj cloud 2026-03-26 08:59:05 +05:30
Debanjum
8965db7087 Bump server, ciient dependencies 2026-03-26 08:59:05 +05:30
lif
8b8504edb8 Fix AttributeError when memories disabled and setting is None (#1296) 
## Summary
- Add null checks for `config.setting` in `get_chat_model()` and
`aget_chat_model()` to prevent `AttributeError` when memories are
disabled
- When the memory toggle creates a `UserConversationConfig` via
`get_or_create` with `setting=None`, accessing
`config.setting.price_tier` crashes — now falls through to the default
chat model instead

## Root Cause
The "Enable Memories" toggle PATCH endpoint uses `get_or_create` on
`UserConversationConfig`, which can create a config with `setting=None`.
Both `get_chat_model()` and `aget_chat_model()` then crash:
- For subscribed users: `if config:` passes but `return config.setting`
returns `None`, causing downstream crashes
- For non-subscribed users: `config.setting.price_tier` raises
`AttributeError` on `None`

## Fix
Change `if config:` → `if config and config.setting:` (subscribed path)
and add `and config.setting` guard before `.price_tier` access
(non-subscribed path), in both sync and async variants.

## Test plan
- [ ] Toggle memories off with no prior chat model configured — settings
page should still load
- [ ] Chat responses should use default model when setting is None
- [ ] Existing users with configured chat models should be unaffected

Fixes #1287

Signed-off-by: majiayu000 <1835304752@qq.com>
 2026-03-26 08:26:25 +05:30
Debanjum
7475a781bc Release Khoj version 2.0.0-beta.27 2026-03-26 01:40:02 +05:30
Debanjum
a19e7acd5a Fix TemplateResponse calls to be compatible with Starlette 1.0.0 
Starlette 1.0.0 removed the deprecated TemplateResponse signature
where `name` was the first positional arg and `request` was passed
inside `context`. The new signature requires `request` as the first
positional argument: TemplateResponse(request, name=...).

This caused a 500 error in production on web client endpoints with:
"Jinja2Templates.TemplateResponse() missing 1 required positional
argument: 'name'" (with older Starlette) or "'request'" (with 1.0.0).

Update all TemplateResponse calls in web_client.py to use the new
Starlette 1.0.0 signature: pass `request` as the first positional
arg and `name` as an explicit keyword argument.

Issue didn't trigger locally as uv is used locally and pip in docker
builds. These resolve dependencies including starletter version to
install differently. Locally 0.52.0 was installed while on production
starlette 1.0.0 was used. This is what caused the issue and the
mismatch in expectation
 2026-03-26 01:38:41 +05:30
Debanjum
b8797e00fa Release Khoj version 2.0.0-beta.26 2026-03-25 20:37:55 +05:30
Debanjum
f356386f3a Ignore dup file errors for pypi wheel validation. Expected Next 15 behavior 2026-03-25 20:30:12 +05:30
Debanjum
d4df9a73ec Use next Link instead of raw a html tags to wrap more links on web app 2026-03-25 20:07:43 +05:30
Debanjum
f7bce48934 Show Khoj cloud deprecation banner on web app to Khoj cloud users 
Add banner to home, chat, shared chat and settings pages for coverage.
Link to settings account section to export data and mention Khoj
self-host option in banner
 2026-03-25 19:31:53 +05:30
Debanjum
b8f82b27f5 Use next Link instead of raw a html tags to wrap Khoj home logo link 2026-03-25 19:14:59 +05:30
Debanjum
a9749c7184 Upgrade to Next.js 15 for web app 2026-03-25 18:32:48 +05:30
Debanjum
51a56af7ca Skip automation tests when GEMINI_API_KEY is not set 
- Add missing skipif decorator to test_create_automation
- Change skip condition from 'is None' to 'not' (falsy check) to
  also handle empty string, which happens when GitHub secrets are
  unavailable in fork PRs
 2026-03-25 18:09:24 +05:30
Debanjum
7264ebf533 Bump package dependencies 
Changes (4 files):
- pyproject.toml: authlib 1.6.6 → 1.6.9
- src/interface/web/package.json: dompurify ^3.2.6 → ^3.3.2, eslint-config-next 14.2.3 → 14.2.35
- documentation/package.json: @docusaurus/* → ^3.9.2, added serialize-javascript resolution

And regenerated lock files.

The only resolution override is serialize-javascript in documentation,
which is unavoidable since Docusaurus still pins old
copy-webpack-plugin and css-minimizer-webpack-plugin that depend on
serialize-javascript ^6.x.
 2026-03-25 18:09:12 +05:30
Tay
0e169159f8 Close leaked file handle in orgnode parser (#1284) 
## Summary

`src/khoj/processor/content/org_mode/orgnode.py:57` opens a file with
`open(filename, "r")` but never closes it. The file handle leaks for the
lifetime of the returned `Orgnode` list.

## Fix

Replaced bare `open()` with a `with` statement to ensure the file is
closed after `makelist()` finishes reading.

```python
# Before
def makelist_with_filepath(filename):
    f = open(filename, "r")
    return makelist(f, filename)

# After
def makelist_with_filepath(filename):
    with open(filename, "r") as f:
        return makelist(f, filename)
```

This is safe because `makelist()` fully consumes the file during the
call (building the Orgnode list from file contents), so the file handle
is no longer needed after it returns.
 2026-03-25 18:03:20 +05:30
BillionToken
530443a4f6 Fix UnboundLocalError in PdfToEntries.extract_text when PDF processing fails (#1292) 
When PyMuPDFLoader fails to process an invalid PDF file, the exception
is caught but pdf_entry_by_pages is referenced before assignment, 
causing an UnboundLocalError.

Initialized pdf_entry_by_pages to an empty list before the try block so 
the return statement always has a valid value, even when an exception
occurs.

Verified with both invalid input (returns []) and valid PDFs (returns
extracted text).

Fixes #1289

Co-authored-by: BillionClaw <267901332+BillionClaw@users.noreply.github.com>
 2026-03-25 17:47:50 +05:30
yang1002378395-cmyk
e863126140 fix: ChatModel.__str__ returns None when friendly_name is null (#1277) 
## Problem
When `ChatModel.friendly_name` is `None`, the `__str__` method returns
`None`, causing:
```
TypeError: __str__ returned non-string (type NoneType)
```

## Solution
Fall back to `name` field when `friendly_name` is `None`.

Related issue: #1251

Co-authored-by: 阳虎 <yanghu@yanghudeMacBook-Pro.local>
 2026-03-20 00:58:44 +05:30
jnMetaCode
678549c6b0 Fix extract_from_webpage discarding pre-fetched content (#1269) 
## Summary

In `extract_from_webpage()`, the `content` parameter is unconditionally
overwritten to `None` on the line before the `is_none_or_empty(content)`
check. This means any pre-fetched content (e.g. text content already
retrieved by the Exa search engine) is always discarded, forcing an
unnecessary re-scrape of the webpage.

## Bug

```python
async def extract_from_webpage(
    url: str,
    subqueries: set[str] = None,
    content: str = None,     # <-- caller passes pre-fetched content
    ...
) -> Tuple[set[str], str, Union[None, str]]:
    content = None            # <-- BUG: immediately overwrites it
    if is_none_or_empty(content):  # always True
        content = await scrape_webpage_with_fallback(url)
```

## Fix

Remove the `content = None` assignment so the passed-in content is used
when available, falling back to scraping only when needed.

This bug was introduced in a refactor and causes:
- Wasted API calls to web scrapers for pages whose content is already
available
- Increased latency for search results that include inline content (e.g.
Exa)

Signed-off-by: JiangNan <1394485448@qq.com>
 2026-03-17 10:33:52 +05:30
jnMetaCode
6735d33af2 Fix operator precedence in research iteration counter (#1271) 
## Summary

Fix a Python operator precedence bug in the `research()` function that
causes `current_iteration` to be set to a boolean instead of the actual
count of previous iterations.

## Bug

```python
if current_iteration := len(previous_iterations) > 0:
```

Python evaluates this as:
```python
if current_iteration := (len(previous_iterations) > 0):  # assigns True or False
```

So `current_iteration` becomes `True` (1) or `False` (0) regardless of
how many previous iterations exist.

## Fix

```python
if (current_iteration := len(previous_iterations)) > 0:
```

With parentheses, `current_iteration` is correctly set to the count
(e.g. 4), and then compared to 0.

## Impact

When resuming research with previous iterations, the loop counter was
effectively reset to 1 instead of the true count. This allowed the
research loop to run significantly more iterations than `MAX_ITERATIONS`
intended, wasting compute and API calls.

Signed-off-by: JiangNan <1394485448@qq.com>
 2026-03-17 10:28:56 +05:30
Olexandr88
0e9878c070 Remove redundant SDK version check in LauncherActivity of Android app (#1263) 
Remove redundant SDK version check in LauncherActivity since both
branches set the same orientation value. This simplifies the code
without changing behavior

Signed-off-by: Olexandr88 <radole1203@gmail.com>
 2026-03-06 12:10:20 +05:30
layla
2c82967807 Fix typos in telemetry error message and comment (#1265) 
Fix spelling typos in telemetry.py. Corrects 'recieved' to 'received'
and 'equest' to 'request' in comments and error messages.
 2026-03-06 12:04:37 +05:30
Debanjum
17be2d4800 Update Pipali project announcement in README 2026-03-06 12:02:50 +05:30
Debanjum
aeea140099 Update What's New in Readme - Mention Pipali release 2026-03-05 21:31:07 -08:00
saba imran
b864cb1f30 only show the payment card on the settings page if the user is subscribed 2026-03-02 10:35:09 -08:00
Debanjum
0b8cf5112f Drop trailing slash to get memories via api on web app in production 
Trailing slash in api calls to server doesn't work in production
behind proxy, only in local next.js dev server.
 2026-02-24 10:46:42 -08:00
Debanjum
94bae4789a Release Khoj version 2.0.0-beta.25 2026-02-22 11:56:01 -08:00
lif
5a51f17a71 Fix AttributeError when Eleven Labs API key is not set (#1238) 
## Summary
- Fixes AttributeError: 'str' object has no attribute 'iter_content' in
text_to_speech endpoint
- When `ELEVEN_LABS_API_KEY` is not configured, the function was
returning a string instead of a Response object

## Changes
- Introduced `TextToSpeechError` exception class in `text_to_speech.py`
- Changed `generate_text_to_speech` to raise exception instead of
returning error string
- Updated API endpoint to catch the exception and return HTTP 501 (Not
Implemented)

## Test plan
- [x] Code passes ruff lint check
- [ ] Manual testing with and without Eleven Labs API key configured

Fixes #1049

---------

Signed-off-by: majiayu000 <1835304752@qq.com>
Co-authored-by: Debanjum <debanjum@gmail.com>
 2026-02-23 01:23:57 +05:30
Koshik Debanath
b0cd8dc8fd Add ability to copy references from Web App (#1144) 
Add a "Copy References" button to the references pane in the web app.

In ReferencePanel Component
- Add a "Copy References" button to the `ReferencePanel` component.
- Implement functionality to copy all references (notes, online, and
code) as a markdown bullet list.
- Update the `TeaserReferencesSection` component to include the "Copy
References" button.
- Show copied to clipboard indicator when references copied on button click

Closes #1021

---------

Co-authored-by: Debanjum <debanjum@gmail.com>
 2026-02-23 01:11:45 +05:30
Sam Ho
0ba0f3d0f2 Show autocomplete suggestions for File Query Filters on Obsidian App (#1128) 
- When you type in search modal, and matches the pattern `file:`, you
should see list of all files in vault and non-vault
- This list is filtered down as you type more letters 


### Technical Details

- Added file filter mode (`isFileFilterMode` state) to filter search
results by specific files
- Updated `getSuggestions()` function to search file from vault and
non-vault via khoj backend.
- Updated the selection behavior to handle both file selection and
search result selection

Closes https://github.com/khoj-ai/khoj/issues/1025

---------

Co-authored-by: Debanjum <debanjum@gmail.com>
 2026-02-23 00:33:17 +05:30
Debanjum
60a2f6d4da Constrain setuptools to resolve installing server in ci 
Error due openai-whisper depending on pkg_resource that setuptools
seems to have dropped.

See https://github.com/pypa/setuptools/issues/5174 for reference

Also bump pillow version
 2026-02-22 10:48:04 -08:00
Debanjum
9dfef3f40b Bump more server dependencies and update uv lock file too 2026-02-22 10:28:53 -08:00
Debanjum
44b9240253 Bump server, desktop, obsidian and docs dependencies 2026-02-22 09:19:50 -08:00
Debanjum
21c51b9ace Ensure only serve home landing page files under specified path 2026-02-22 09:19:41 -08:00
Debanjum
9cbf620e45 Retry (with fallback) on Gemini fails with internal server error 
Khoj should use a fallback model when available to retry request if
calls to Gemini fail with internal server error.
 2026-01-06 12:14:08 -08:00
Henri Jamet
1fd6e16cff Improve Obsidian Batch Sync. Show Progress, Storage Used on Settings Page (#1221) 
### **feat(obsidian): Enhance Sync Experience with Progress Bars and Bug
Fixes**

This pull request significantly improves the content synchronization
experience for Obsidian users by fixing a critical bug and introducing
new UI elements for better feedback and monitoring.

The previous implementation could fail with `403 Forbidden` errors when
syncing a large number of files due to server-side rate limiting. This
update addresses that issue and provides users with clear, real-time
feedback on storage usage and sync progress.

---
### Key Changes

* **Improve Sync Robustness**
Refactor `updateContentIndex` to sync files prioritized by file type (md
> pdf > image) and batched by size (10Mb) and item limits (50 items).
This respects server rate limits and ensures that large vaults can be
indexed reliably without triggering `403` errors.

* **Show Cloud Storage Usage Bar**
A progress bar has been added to the settings page to display cloud
storage usage.
* **Total Limit**: The storage limit (**10 MB** for free, **500 MB** for
premium) is now reliably determined by the `is_active` flag returned
from the `/api/v1/user` endpoint, eliminating fragile client-side
heuristics.
* **Used Space**: The used space is calculated via a **client-side
estimation** of all files configured for synchronization. This provides
a clear and immediate indicator of the vault's storage footprint.

* **Show Real-time Sync Progress Bar**
When a manual sync is triggered via the "Force Sync" button, a progress
bar now appears, providing real-time feedback on the operation.
* It displays the number of files processed against the total number of
files to be indexed or deleted.
* This is implemented using a **callback mechanism** (`onProgress`) to
cleanly communicate progress from the sync logic (`utils.ts`) to the UI
(`settings.ts`) without coupling them.

* **Auto-refresh Storage Used After Sync**
The Cloud Storage Usage bar is now automatically refreshed upon the
completion of a "Force Sync". This ensures the user immediately sees the
updated storage estimation without needing to reopen the settings panel.

---
### Visuals

<img width="980" height="237" alt="image"
src="https://github.com/user-attachments/assets/2b3ce420-766b-476f-9fc0-c6b38c0226fb"
/>

---------

Co-authored-by: Debanjum <debanjum@gmail.com>
 2026-01-03 13:04:46 +05:30
sabaimran
d6c2d1fa49 Give Khoj Long Term Memories (#1168) 
# Motivation
A major component of useful AI systems is adaptation to the user
context. This is a major reason why we'd enabled syncing knowledge
bases. The next steps in this direction is to dynamically update the
evolving state of the user as conversations take place across time and
topics. This allows for more personalized conversations and to maintain
context across conversations.

# Overview
This change introduces medium and long term memories in Khoj. 
- The scope of a conversation can be thought of as short term memory. 
- Medium term memory extends to the past week.
- Long term memory extends to anytime in the past, where a search query
results in a match.

# Details
- Enable user to view and manage agent generated memories from their
settings page
- Fully integrate the memory object into all downstream usage, from
image generation, notes extraction, online search, etc.
- Scope memory per agent. The default agent has access to memories
created by other agents as well.
- Enable users and admins to enable/disable Khoj's memory system

---------

Co-authored-by: Debanjum <debanjum@gmail.com>
 2026-01-03 09:07:05 +05:30
Debanjum
d55a00288b Release Khoj version 2.0.0-beta.24 2026-01-01 19:59:10 -08:00
Debanjum
ff4b9f3502 Render mermaid diagram wrapped in markdown codeblocks on web app 2026-01-01 19:57:49 -08:00
Debanjum
19900e42ef Fix registering subscription payment failures 2026-01-01 19:57:49 -08:00
Debanjum
a58ae3dd84 Make LLM actors write & code sandbox check for artifacts in /home/user 
Fix
- Ensure researcher and coder know to save files to /home/user dir
- Make E2B code executor check for generated files in /home/user
- Do not re-add file types already downloaded from /home/user

Issues
- E2B has a mismatch in default home_dir for run_code & list_dir cmds
So run_code was run with /root as home dir. And list_dir("~") was
checking under /home/user. This caused files written to /home/user
by code not to be discovered by the list_files step.
- Previously the researcher did not know that generated files should
be written to /home/user. So it could tell the coder to save files to
a different directory. Now the researcher knows where to save files to
show them to user as well.
 2025-12-29 14:57:37 -08:00
Debanjum
b607a6187e Release Khoj version 2.0.0-beta.23 2025-12-29 01:42:25 -08:00
Boris Smus
f413ce7354 Enable excluding folders to sync from obsidian plugin settings (#1235) 
- Add excludeFolders field to KhojSetting interface
- Rename 'Sync Folders' to 'Include Folders' for clarity
- Add 'Exclude Folders' UI section with folder picker
- Filter out excluded folders during content sync
- Show file counts when syncing (X of Y files)
- Prevent excluding root folder

This allows users to exclude specific directories (e.g., Inbox,
Highlights) from being indexed, while the existing Include Folders acts
as a whitelist.

---------

Co-authored-by: Debanjum <debanjum@gmail.com>
 2025-12-29 15:09:19 +05:30
Debanjum
9b9cdc756f Capture more files generated by code execution in sandbox 
This change had been removed in 9a8c707 to avoid overwrites. We now
use random filename for generated files to avoid overwrite from
subsequent runs.

Encourage model to write code that writes files in home folder to
capture with logical filenames.
 2025-12-29 00:57:17 -08:00
Debanjum
c5650f166a Make Nano Banana Pro output 2K resolution images 2025-12-29 00:57:17 -08:00
Debanjum
1b7ccd141d Harden the user check of the Notion integration 2025-12-29 00:57:17 -08:00
Debanjum
b8eeefa0b1 Bump web app package dependencies 2025-12-29 00:57:17 -08:00
Debanjum
9801ffd2de Add Khoj app landing page. Show it when unauthenticated users open app 
Add khoj app landing page to khoj monorepo. Show in a more natural
place, when non logged in users open the khoj app home page.

Authenticated users still see logged in home page experience.
 2025-12-29 00:57:17 -08:00
Debanjum
5e65754a8b Unify login via popup on home. No need for separate login html page. 
Delete old login html page. Login via popup on home is the single,
unified login experience.

Have docs mention khoj home url, no need to mention /login as login
popup shows on home page too
 2025-12-29 00:57:17 -08:00
Debanjum
f65f6ae848 Fix streaming thoughts from multi-turn tools after parallel tool calling 
Single turn tools are still executed in parallel. Multi turn tools
like operator are executed in serial.
 2025-12-29 00:57:17 -08:00
Debanjum
446a23524c Remove trial subscriptions from Khoj cloud 2025-12-16 16:23:47 -08:00
Debanjum
cdcbdf8459 Execute tool calls in parallel to make research iterations faster 2025-12-13 22:55:54 -08:00
Debanjum
054ed79fdf Allow LLMs to make parallel tool call requests 
Why
--
- The models are now smart enough to usually understand which tools to
  call in parallel and when.

- The LLM can request more work for each call to it, which is usually
  the slowest step. This speeds up work by reearch agent. Even though
  each tool is still executed in sequence (for now).
 2025-12-13 21:37:49 -08:00
Debanjum
f4c519a9d0 Release Khoj version 2.0.0-beta.22 2025-12-07 20:32:28 -08:00
Debanjum
6480e99266 Upgrade server, documentation dependencies 2025-12-07 20:29:11 -08:00
Debanjum
181332dcb8 Improve Claude context caching to improve response cost, intelligence 
Old thought messages are dropped by default by the Anthropic API. This
change ensures old thoughts are kept. This should improve cache
utilization to reduce costs. And keeping old thoughts may also improve
model intelligence.
 2025-12-07 20:23:58 -08:00
Debanjum
9c03af2735 Disable parallel tool call by anthropic models as unsupported 
Khoj doesn't handle parallel tool calling right now. Models were told
to call tools in serial but it wasn't enforced via the Anthropic API.
So if model did try make parallel tool call, next response would fail
as it expects a tool result for the other tool calls. But khoj just
returned the first tool calls results. This mostly affected haiku due
to its lower fine-grained instruction following capabilities.

This changes enforces serial tool calls at the API layer to avoid this
issue altogether for claude models.
 2025-12-07 20:23:58 -08:00
Debanjum
4654ac4962 Enable reasoning for Claude Haiku 4.5. Track costs of Opus 4.5 2025-12-07 20:23:58 -08:00
Debanjum
f1337c3b07 Handle unset event in chat response stream 
Testing unset event field is an edge case that would unnecessarily
prematurely terminate stream. Ignoring it stabilizes response stream
completion.
 2025-12-07 20:23:58 -08:00
Debanjum
bdf9afa726 Set openai api output tokens high by default to not hit length limits 
Explicitly set completion tokens high to avoid early termination
issues, especially when trying to generate structured responses.
 2025-12-07 20:23:58 -08:00
Debanjum
5b6dab1627 Use consistent summarizer result for failed research iteration 2025-12-05 10:02:08 -08:00
Debanjum
3941159bd6 Release Khoj version 2.0.0-beta.21 2025-11-29 17:31:14 -08:00
Debanjum
e2340c709f Fix extracting image by URL in chat history when using Nano Banana Pro 
Logical error due to else conditional being not correctly indented.
This would result in error in using gemini 3 pro image when images are
in S3 bucket.
 2025-11-29 17:08:17 -08:00
Debanjum
856864147b Drop image generation support for Stability AI models 
Reduces maintenance burden by dropping support for old ai model
providers
 2025-11-29 17:08:11 -08:00
Debanjum
c41e37d734 Release Khoj version 2.0.0-beta.20 2025-11-29 16:12:38 -08:00
Debanjum
731700ac43 Support fallback deep, fast chat models via server chat settings 
Overview
---
This change enables specifying fallback chat models for each task
type (fast, deep, default) and user type (free, paid).

Previously we did not fallback to other chat models if the chat model
assigned for a task failed.

Details
---
You can now specify multiple ServerChatSettings via the Admin Panel
with their usage priority. If the highest priority chat model for the
task, user type fails, the task is assigned to a lower priority chat
model configured for the current user and task type.

This change also reduces the retry attempts for openai chat actor
models from 3 to 2 as:
- multiple fallback server chat settings can now be created. So
  reducing retries with same model reduces latency.
- 2 attempts is inline with retry attempts with other model
  types (gemini, anthropic)
 2025-11-29 15:57:35 -08:00
Debanjum
99f16df7e2 Use fast model in default mode and for most chat actors 
What
--
- Default to using fast model for most chat actors. Specifically in this
  change we default to using fast model for doc, web search chat actors
- Only research chat director uses the deep chat model.
- Make using fast model by chat actors configurable via func argument

Code chat actor continues to use deep chat model and webpage reader
continues to use fast chat model.

Deep, fast chat models can be configured via ServerChatSettings on the
admin panel.

Why
--
Modern models are good enough at instruction following. So defaulting
most chat actor to use the fast model should improve chat speed with
acceptable response quality.

The option to fallback to research mode for higher quality
responses or deeper research always exists.
 2025-11-29 15:57:35 -08:00
Debanjum
da493be417 Support image generation with Gemini Nano Banana 2025-11-29 15:57:35 -08:00
Debanjum
dd4381c25c Do not try render invalid image paths in message on web app 
Avoids rendering flicker from attempt to render invalid image paths
referenced in message by khoj on web app.

The rendering flicker made it very annoying to interact with
conversations containing such messages on the web app.

The current change does lightweight validation of image url before
attempting to render it. If invalid image url detected, the image is
replaced with just its alt text.
 2025-11-29 15:23:51 -08:00
Debanjum
51b893d51d Fix to ensure rectangular generated images are not cropped on web app 
Previously non-square images would get cropped when being displayed on
web app
 2025-11-29 15:23:51 -08:00
Debanjum
32966646e2 Avoid ai hover summaries in vscode dev env for now 2025-11-29 15:23:51 -08:00
Debanjum
043777c1bd Release Khoj version 2.0.0-beta.19 2025-11-18 15:30:50 -08:00
Debanjum
47a55c20a0 Associate folder icon with all doc tools use in thinking UX on web app 
The newer grep_files and list_files should also be associated with
document search in train of thought visualization on the web app.
 2025-11-18 15:17:38 -08:00
Debanjum
6459150870 Upgrade packages for documentation and desktop app 2025-11-18 15:17:38 -08:00
Debanjum
03dad1348a Support Minimax M2. Extract its thinking from response 
- Use qwen style <think> tags to extract Minimax M2 model thoughts
- Use function to mark models that use in-stream thinking (including
  Kimi K2 thinking)
 2025-11-18 14:13:28 -08:00
Debanjum
57d6ebb1b8 Support Google Gemini 3 
- Use thinking level for gemini 3 models instead of thinking budget.
- Bump google gemini library
- Add default context, pricing
 2025-11-18 14:13:24 -08:00
Debanjum
a30c5f245d Skip non-serializable, binary content parts when token counting 2025-11-18 12:42:18 -08:00
Debanjum
ec31df7154 Test khoj.el with more recent emacs versions 2025-11-18 10:29:02 -08:00
Debanjum
895af42039 Fix unbound response var exception in agent safety checker 2025-11-18 10:29:02 -08:00
Debanjum
748a4f9941 Release Khoj version 2.0.0-beta.18 2025-11-16 11:08:44 -08:00
Debanjum
3496189618 Support using MCP tools in research mode 
- Server admin can add MCP servers via the admin panel
- Enabled MCP server tools are exposed to the research agent for use
- Use MCP library to standardize interactions with mcp servers
  - Support SSE or Stdio as transport to interact with mcp servers
  - Reuse session established to MCP servers across research iterations
 2025-11-16 10:50:30 -08:00
Debanjum
2ac7359092 Simplify webpage read function names and drop unused return args 2025-11-16 10:50:30 -08:00
Debanjum
f1a34f0c2a Prefer Exa for web search over Google, Firecrawl 
Google and Firecrawl do not provide good web search descriptions (within
given latency requirements). Exa does better than them.

So prioritize using Exa over Google or Firecrawl when multiple web
search providers available.
 2025-11-16 10:50:30 -08:00
Debanjum
45f4253120 Move Olostep scraping config into its webpage reader for cleaner code 2025-11-16 10:50:30 -08:00
Debanjum
e6a5d3dc3d Deprecate support for using Firecrawl webpage summarizer 
Better speed and control by using Khoj webpage summarizer. Reduce code
cruft by clearing unused features.
 2025-11-16 10:50:30 -08:00
Debanjum
0415b31a23 Upgrade Firecrawl web provider to use their v2 api 2025-11-16 10:50:30 -08:00
Debanjum
61cb2d5b7e Enable webpage reading with Exa. Remove Jina web page reader 
Support using Exa for webpage reading. It seems much faster than
currently available providers.

Remove Jina as a webpage reader and remaining references to Jina from
code, docs. It was anyway slow and API may shut down soon (as it was
bought by Elastic).

Update docs to mention Exa for web search and webpage reading.
 2025-11-16 10:50:30 -08:00
Debanjum
d57c597245 Refactor count_tokens, get_encoder methods to utils/helper.py 
Simplify get_encoder to not rely on global state. The caching
simplification is not necessary for now.
 2025-11-16 10:50:30 -08:00
Debanjum
15482c54b5 Fix type of count total tokens system_message argument 2025-11-16 10:50:30 -08:00
Debanjum
761af5f98c Drop spurious results close xml tag in research shared with llm 2025-11-16 10:50:30 -08:00
Debanjum
1f3c1e1221 Remove spurious comments in desktop app chatutils.js 2025-11-16 10:50:30 -08:00
Debanjum
8490f2826b Reduce evaluator llm verbosity during eval 2025-11-16 10:50:30 -08:00
Debanjum
630ce77b5f Improve agent update/create safety check. Make reason field optional 
Issue
---
When agent personality/instructions are safe, we do not require the
safety agent to give a reason. The safety check agent was told this in
the prompt but it was not reflected in the json schema being used.

Latest openai library started throwing error if response doesn't match
requested json schema.

This broke creating/updating agents when using openai models as safety
agent.

Fix
---
Make reason field optional.

Also put send_message_to_model_wrapper in try/catch for more readable
error stacktrace.
 2025-11-14 06:47:51 -08:00
Debanjum
cbeb220f00 Show validation errors in UX if agent creation, update fails 
Previously we only showed unsafe prompt errors to user when
creating/updating agent. Errors in name collision were not shown on
the web app ux.

This change ensures that such validation errors are bubbled up to the
user in the UX. So they can resolve the agent create/update error on
their end.
 2025-11-12 10:51:07 -08:00
Debanjum
d7e936678d Release Khoj version 2.0.0-beta.17 2025-11-11 16:38:46 -08:00
Debanjum
4556773f42 Improve support for new kimi k2 thinking model 
Recognize thinking by kimi k2 thinking model in <think> xml blocks
 2025-11-11 16:20:41 -08:00
Debanjum
2c54a2cd10 Improve web browsing train of thought status text shown on web app 2025-11-11 16:14:27 -08:00
Debanjum
aab0653025 Add price for grok 4, grok 4 fast for cost estimatation 2025-11-11 16:14:19 -08:00
Debanjum
b14e6eb069 Count cache, reasoning tokens to estimate cost for models served over openai api 
Count cached tokens, reasoning tokens for better cost estimates for
models served over an openai compatible api. Previously we didn't
include cached token or reasoning tokens in costing.
 2025-11-11 16:12:48 -08:00
Debanjum
ce6d75e5a2 Support Exa as web search provider 2025-11-11 16:12:48 -08:00
Debanjum
5760f3b534 Drop support for web search, read using Jina as provider 
There are faster, better web search, webpage read providers. Only keep
reasonable quality online context providers.

Jina was good for self-hosting quickstart as it provided a free api
key without login. It does not provide that now. Its latencies are
pretty high vs other online context providers.
 2025-11-11 16:12:48 -08:00
Debanjum
c022e7d553 Upgrade Anthropic Operator editor version 2025-11-11 16:12:48 -08:00
Debanjum
88a1fc75cc Track cost of claude haiku 4.5 model 2025-11-11 16:12:48 -08:00
Debanjum
a809de8970 Handle skip indexing of unsupported image files 
Previously unsupported image file types would trigger an unbound local
variable error.
 2025-11-11 16:09:08 -08:00
Debanjum
749bbed23d Track cost of claude sonnet 4.5 models 2025-11-11 16:09:04 -08:00
Debanjum
69cceda9ab Bump server dependencies 2025-11-11 16:08:38 -08:00
Debanjum
140a3ef943 Avoid unbound chunk variable error in ai api call from completion func 2025-11-11 16:08:38 -08:00
Debanjum
f2e0b62217 Remove unused default source from default tool picker prompt, help msg 2025-11-11 16:08:37 -08:00
Debanjum
5ef3a3f027 Remove unused eval workflow config to auto read webpage in default mode 2025-09-16 14:55:06 +05:30
Debanjum
6ac2280e41 Release Khoj version 2.0.0-beta.16 2025-09-16 14:47:14 +05:30
Debanjum
1179a4c8f8 Update dev docs to suggest using bun instead of yarn for web app 
Resolves #1218
 2025-09-16 14:13:34 +05:30
Debanjum
51e5c86fcc Bump desktop app dependencies 2025-09-16 14:03:58 +05:30
Debanjum
534ee32664 Bump web app dependencies 2025-09-16 14:03:58 +05:30
Debanjum
e854c1a5a8 Bump django, langchain python server dependencies 2025-09-16 14:03:58 +05:30
Debanjum
2fdb1fcc93 Remove unsupported tool schema fields minimum, maximum for groq api 
Groq API has stopped support minimum and maximum items fields from
tool schema. This unexpectedly broke using AI models served via Groq
API like Kimi K2 and GPT-OSS in research mode.

Improve typing of relevant fields
 2025-09-16 14:03:29 +05:30
Debanjum
3e699e5476 Fix date time rendering when print conversation on web app 2025-09-01 07:09:32 -07:00
Debanjum
0bd4bf182c Show shared chats without login popup shown to unauthenticated users 
The login popup is an unnecessary distraction as you do not need
to be logged in to view shared chats.
 2025-08-31 23:40:09 -07:00
Debanjum
52b1928023 Make gpqa answer evaluator more versatile at extracting mcq answers 2025-08-31 23:40:09 -07:00
Debanjum
703e189979 Deterministically shuffle dataset for consistent data in a eval run 
Previously eval run across modes would use different dataset shuffles.

This change enables a strict apples to apples perf comparison of the
different khoj modes across the same (random) subset of questions by
using a dataset seed per workflow run to sample questions
 2025-08-31 23:40:08 -07:00
Debanjum
edf9ea6312 Release Khoj version 2.0.0-beta.15 2025-08-31 13:21:58 -07:00
Debanjum
d53ede604c Only enable web search with Searxng if KHOJ_SEARXNG_URL env var set 
Instead of implicitly defaulting to assuming it is available as:
- For pip install searxng has to be explicitly setup to work
- For docker install we explicitly do set it up and set the
  KHOJ_SEARXNG_URL env var already

Also check if Searxng URL is also unset before disable web search
tools now that it is required explicit enablement.
 2025-08-31 13:17:05 -07:00
Debanjum
7533e3eecf Use prompt cache key to improve cache hits with openai responses api 
Using prompt cache key enables sticky routing to openai servers.
This increases probability of a chat actor hitting same server and
reusing cached prompts.

We use stable hash of first N characters to uniquely identify a chat
actor prompt
 2025-08-31 12:44:38 -07:00
Debanjum
3c1948e9de Disable code sandbox if no code sandbox configured by admin 
Either set the Terrarium sandbox url or the E2B api key to enable code
sandbox
 2025-08-31 10:15:14 -07:00
Debanjum
3441783d5b Disable web search tool if no search engine configured by admin 
Webpage read is gated behind having a web search engine configured for
now. It can later be decoupled from web search and depend on whether
any web scrapers is configured.
 2025-08-30 00:22:26 -07:00
Debanjum
3aa6f8ba1f Add gemini cached tokens costs for more accurate cost tracking 2025-08-29 15:55:07 -07:00
Debanjum
0babab580a Avoid null ref error when no organic online search results found 2025-08-29 15:54:06 -07:00
Debanjum
00f0d23224 Fix indexing Github, Notion content by linking embeddings model on init 2025-08-29 15:54:06 -07:00
Debanjum
81c651b5b2 Fix truncation tests to check output chat history for truncation 
New truncation logic return a new message list.
It does not update message list by reference/in place since 8a16f5a2a.
So truncation tests should run verification on the truncated chat
history returned by the truncation func instead of the original chat
history passed into the truncation func.
 2025-08-28 15:50:32 -07:00
Debanjum
c0f192b436 Set minimum table width on web app for better readability 2025-08-28 01:58:04 -07:00
Debanjum
dd8e805cfe Add support for Cerebras ai model api 
- It does not support strict mode for json schema, tool use
- It likes text content to be plain string, not nested in a dictionary
- Verified to work with gpt oss models on cerebras
 2025-08-28 01:57:39 -07:00
Debanjum
0a5a882e54 Check if openai compatible ai api supports the responses api endpoint 
Responses API is starting to get supported by other ai apis as well.
This change does preparatory improvements to ease moving to use
responses api with other ai apis.

Use the new, better named `supports_responses_api' method.
The method currently just maps to `is_openai_api'. It will add other
ai apis once support for using responses api with them is added.
 2025-08-28 01:38:47 -07:00
Debanjum
9395c17f34 Fix openai reasoning model handling 
- Fix identifying gpt-oss as openai reasoning model
- Drop unsupported stop param for openai reasoning models
- Drop the Formatting re-enabled logic for openai reasoing only models
  We use responses api for openai models and latest openai models are
  hybrid models, they don't seem to need this convoluted system
  message to format response as markdown
 2025-08-28 01:38:47 -07:00
Debanjum
be79b8a633 Drop unused arguments to default tool picker, research mode 
is_automated_task check isn't required as automation cannot be created
via chat anymore.

conversation specific file_filters are extracted directly in document
search, so doesn't need to be passed down from chat api endpoint
 2025-08-27 14:37:28 -07:00
Debanjum
9d7adbcbaa Pass user attached images to default tool picker for informed selection 
Previously we were just passing placeholder informing the default mode
tool picker that images were attached.
 2025-08-27 14:37:10 -07:00
Debanjum
2091044db5 Prefer agent chat model to extract document search queries 
Make chat model preference order for document search consistent with
all other tools.
 2025-08-27 13:55:50 -07:00
Debanjum
7a42042488 Share context builder for chat final response across model types 
The context building logic was nearly identical across all model
types.

This change extracts that logic into a shared function and calls it
once in the `agenerate_chat_response', the entrypoint to the converse
methods for all 3 model types.

Main differences handled are
- Gemini system prompt had additional verbosity instructions. Keep it
- Pass system messsage via chatml messages list to anthropic, gemini
  models as well (like openai models) instead of passing it as
  separate arg to chat_completion_* funcs.

  The model specific message formatters for both already extract
  system instruction from the messages list. So system messages wil be
  automatically extracted from the chat_completion_* funcs to pass as
  separate arg required by anthropic, gemini api libraries.
 2025-08-27 13:48:33 -07:00
Debanjum
02e220f5f5 Pass args to context builder funcs grouped consistently 
Put context params together, followed by model params
Use consistent ordering to improve readability
 2025-08-27 13:45:36 -07:00
Debanjum
4976b244a4 Set fast, deep think models for intermediary steps via admin panel 
Overview
Enable improving speed and cost of chat by setting fast, deep think
models for intermediate steps and non user facing operations.

Details
- Allow decoupling default chat models from models used for
  intermediate steps by setting server chat settings on admin panel
- Use deep think models for most intermediate steps like tool
  selection, subquery construction etc. in default and research mode
- Use fast think models for webpage read, chat title setting etc.
  Faster webpage read should improve conversation latency
 2025-08-27 13:45:36 -07:00
Debanjum
a99eb841ff Do not search documents when default tool selected by agent 
What
Explicit selection of notes tool/conversation command by agent is
required now.

Why
- Newer models are good at deciding when to look up notes
- Modern khoj is less of a notes only chat to search notes by default
 2025-08-27 13:45:28 -07:00
Debanjum
a52a06ad9d Prefer olostep over firecrawl for webpage read by default 
Default to Olostep as faster and higher webpage read success rate.
Fallback logic will use Firecrawl if Olostep fails.
 2025-08-27 13:45:09 -07:00
Debanjum
e150dc5a91 Improve copying message with math, file links to clipboard on web app 2025-08-27 13:45:09 -07:00
Debanjum
15d1f39d0b Improve instruction to ai model for writing math expressions in LaTeX 2025-08-27 13:45:09 -07:00
Debanjum
05dbb6a7c1 Drop unused generated_files arg from chat context 
generated_files wasn't being set (anymore?). But it was being passed
around through for chat context and being saved to db.

Also reduce variables used to set mermaid diagram description
 2025-08-27 13:45:09 -07:00
Debanjum
8a16f5a2af Reduce logical complexity of constructing context from chat history 
- Process chat history in default order instead of processing it in
  reverse. Improve legibility of context construction for minor
  performance hit in dropping message from front of list.
- Handle multiple system messages by collating them into list
- Remove logic to drop system role for gemma-2, o1 models. Better to
  make code more readable than support old models.
 2025-08-27 13:43:10 -07:00
Debanjum
1e81b51abc Support generating images with different aspect ratios 
You can now specify shape of images to be generated. It can be one of
portrait, landscape or square.
 2025-08-27 13:43:10 -07:00
Debanjum
5a2cae3756 Improve, simplify image generation prompts and context flow 
Use seed to stabilize image change consistency across turns when
- KHOJ_LLM_SEED env var is set
- Using Image models via Replicate
  OpenAI, Google do not support image seed
 2025-08-27 13:43:10 -07:00
Debanjum
0fb6020f30 Remove model type check to construct structured messages 
All model types use a normalized, chatml structured message format
This check isn't used since offline model support was dropped.
 2025-08-27 13:43:10 -07:00
Debanjum
386a17371d Fix identifying deepseek r1 model to process its thinking tokens 2025-08-27 13:43:04 -07:00
Debanjum
ff004d31ef Fix extracting inferred queries from chat history db 
Inferred queries is stored with underscore in db but aliased with - in memory.

This conversation.messages logic was broken, so inferred queries field
of chat message history was getting ignored.

This change fixes that issue and improve previous image generation
description for better context for subsequent image generation attempts.
 2025-08-25 14:19:27 -07:00
Debanjum
892e4d4077 Fix system prompt construction for gemini models 
System prompt was duplicating instructions for gemini models
previously
 2025-08-24 18:22:31 -07:00
Debanjum
00c5aec614 Release Khoj version 2.0.0-beta.14 2025-08-23 12:12:33 -07:00
Debanjum
b99ccbc4c3 Improve table styling, fix chat sidebar height on web app 2025-08-23 02:05:50 -07:00
Debanjum
29ae476a26 Use groq with service tier auto to fallback to flex on rate limit 
Merge gpt-oss config with openai reasoning config as similar tuning.
Add pricing for gpt oss 20b model
 2025-08-23 02:05:50 -07:00
Debanjum
c89c5c7b46 Reorder ai model api columns on admin panel for readability 2025-08-23 01:40:05 -07:00
Debanjum
464c1546b7 Support deepseek v3.1 via official deepseek api 
The new deepseek-chat is powered by deepseek v3.1, which is a hybrid
reasoning model unlike it's predecessor, deepseek v3.
 2025-08-23 01:40:05 -07:00
Debanjum
40488b3b68 Remove redundant exception for retry calls to gemini api 
httpx ReadError inherits from NetworkError so not required to mention
it explicitly in gemini api call retry check
 2025-08-23 00:48:10 -07:00
Debanjum
8aa9c0f534 Reduce max reasoning tokens for gemini models 
A high reasoning tokens does not seem to help for standard Khoj use
cases. And hopefully reducing it may avoid repetition loops by model.
 2025-08-23 00:48:10 -07:00
Debanjum
2823c84bb4 Default to gemini 2.5 model series on init and for eval 2025-08-22 20:34:38 -07:00
Debanjum
c53a70c997 Share debug logs from github eval run for debugging 2025-08-22 19:06:37 -07:00
Debanjum
e2f377c27b Render file reference as link with file preview on hover/click in web app 
Overview
- Khoj references files it used in its response as markdown links.
  For example [1](file://path/to/file.txt#line=121)
- Previously these file links were just shown as raw text
- This change renders khoj's inline file references as a proper links
  and shows file content preview (around specified line if deeplink)
  on hover or click in the web app

Details
- Render inline file references as links in chat message on web app.
  Previously references like [1](file://path/to/file.txt#line=120)
  would be shown as plain text. Now they are rendered as links
- Preview file content of referenced files on click or hover.
  If reference uses a deeplink with line number, the file content
  around that line is shown on hover, click. Click allows viewing file
  preview on mobile, unlike hover. Hover is easier with mouse.
 2025-08-22 18:24:27 -07:00
Debanjum
d8b7e9c8a5 Handle unset content type key when indexing knowledge base on server 2025-08-22 18:24:27 -07:00
Debanjum
3c3205bb06 Fix and improve file read, write handling in Obsidian 
Fixes
- Fix to allow khoj to delete content in obsidian write mode
- Do not throw error when no edit blocks in write mode on obsidian
- Limit retries to fix invalid edit blocks in obsidian write mode

Improvements
- Only show 3 recent files as context in obsidian file read, write mode
- Persist open file access mode setting across restarts in obsidian
- Make khoj obsidian keyboard shortcuts toggle voice chat, chat history
- Do not show <SYSTEM> instructions in chat session title on obsidian

Closes #1209
 2025-08-20 20:20:12 -07:00
Debanjum
48ed7afab8 Do not show <SYSTEM> instructions in chat session title on obsidian 
In obsidian we have a hacky system instruction being passed in read,
write file access modes. This shouldn't be shown in chat sessions list
during view or edit. It is an internal implementation detail.
 2025-08-20 20:18:27 -07:00
Debanjum
82dc7b115b Fix to allow khoj to delete content in obsidian write mode 
Previous regex and replacement logic did not allow replace block to be
empty
 2025-08-20 20:18:27 -07:00
Debanjum
7645cbea3b Do not throw error when no edit blocks in write mode on obsidian 
Editing is an option, not a requirement in file write/edit mode.
 2025-08-20 20:18:27 -07:00
Debanjum
2e6928c582 Limit retries to fix invalid edit blocks in obsidian write mode 2025-08-20 20:18:27 -07:00
Debanjum
c5e2373d73 Make khoj obsidian keyboard shortcuts toggle voice chat, chat history 
Previously hitting voice chat keybinding would just start voice chat,
not end it and just open chat history and not close it.

This is unintuitive and different from the equivalent button click
behaviors.

Fix toggles voice chat on/off and shows/hides chat history when hit
Ctrl+Alt+V, Ctrl+Alt+O keybindings in khoj obsidian chat view
 2025-08-20 20:18:27 -07:00
Debanjum
d8b2df4107 Only show 3 recent files as context in obsidian file read, write mode 
Related #1209
 2025-08-20 20:18:27 -07:00
Debanjum
eb2f0ec6bc Persist open file access mode setting across restarts in obsidian 
Allows a lightweight mechanism to persist this user preference.
Improve hover text a bit for readability.

Resolves #1209
 2025-08-20 20:18:27 -07:00
Debanjum
2884853c98 Make plugin object accessible to chat, find similar panes in obsidian 
Allows ability to access, save settings in a cleaner way
 2025-08-20 20:18:27 -07:00
Debanjum
9f6aa922a2 Improve Khoj research tools, gpt-oss support and ai api usage 
Better support for GPT OSS
- Tune reasoning effort, temp, top_p for gpt-oss models
- Extract thoughts of openai style models like gpt-oss from api response

Tool use improvements
- Improve view file, code tool prompts. Format other research tool prompts
- Truncate long words in code tool stdout, stderr for context efficiency
- Use instruction instead of query as code tool argument
- Simplify view file tool. Limit viewing upto 50 lines at a time
- Make regex search tool results look more like grep results
- Update khoj personality prompts with better style, capability guide

Web UX improvements
- Wrap long words in train of thought shown on web app
- Do not overwrite charts created in previous code tool use during research
- Update web UX when server side error or hit stop + no task running

Fix AI API Usage
- Use subscriber type specific context window to generate response
- Fix max thinking budget for gemini models to generate final response
- Fix passing temp kwarg to non-streaming openai completion endpoint
- Handle unset reasoning, response chunk from openai api while streaming
- Fix using non-reasoning openai model via responses API
- Fix to calculate usage from openai api streaming completion
 2025-08-20 20:06:18 -07:00
Debanjum
13d26ae8b8 Wrap long words in train of thought shown on web app 2025-08-20 19:07:28 -07:00
Debanjum
fb0347a388 Truncate long words in stdout, stderr for context efficiency 
Avoid long base64 images etc. in stdout, stderr to result in context
limits being hit.
 2025-08-20 19:07:28 -07:00
Debanjum
dbc3330610 Tune reasoning effort, temp, top_p for gpt-oss models 2025-08-20 19:07:28 -07:00
Debanjum
83d725d2d8 Extract thoughts of openai style models like gpt-oss from api response 
They use delta.reasoning instead of delta.reasoning_content to share
model reasoning
 2025-08-20 19:07:28 -07:00
Debanjum
f483a626b8 Simplify view file tool. Limit viewing upto 50 lines at a time 
We were previously truncating by characters. Limiting by max lines
allows model to control line ranges they request
 2025-08-20 19:07:28 -07:00
Debanjum
f5a4d106d1 Use instruction instead of query as code tool argument 2025-08-20 19:07:28 -07:00
Debanjum
c5a9c81479 Update khoj personality prompts with better style, capability guide 
- Add more color to personality and communication style
- Split prompt into capabilities and style sections
- Remove directives in personality meant for older, less smart models.
- Discourage model from unnecessarily sharing code snippets in final
  response unless explicitly requested.
 2025-08-20 19:07:28 -07:00
Debanjum
2c91edbb25 Improve view file, code tool prompts. Format other research tool prompts 2025-08-20 19:07:28 -07:00
Debanjum
452c794e93 Make regex search tool results look more like grep results 2025-08-20 19:07:28 -07:00
Debanjum
9a8c707f84 Do not overwrite charts created in previous code tool use during research 2025-08-20 19:07:28 -07:00
Debanjum
e0007a31bb Update web UX when server side error or hit stop + no task running 
- Ack websocket interrupt even when no task running
  Otherwise chat UX isn't updated to indicate query has stopped
  processing for this edge case

- Mark chat request as not being procesed on server side error
 2025-08-20 19:07:28 -07:00
Debanjum
222cc19b7f Use subscriber type specific context window to generate response 2025-08-20 19:07:28 -07:00
Debanjum
ff73d30106 Fix max thinking budget for gemini models to generate final response 2025-08-20 19:07:28 -07:00
Debanjum
34dca8e114 Fix passing temp kwarg to non-streaming openai completion endpoint 
It is already being passed in model_kwargs, so not required to be
passed explicitly as well.

This code path isn't being used currently, but better to fix for
if/when it is used
 2025-08-20 19:07:28 -07:00
Debanjum
8862394c15 Handle unset reasoning, response chunk from openai api while streaming 2025-08-20 19:07:28 -07:00
Debanjum
14b4d4b663 Fix using non-reasoning openai model via responses API 
Pass arg to include encrypted reasoning only for reasoning openai
models. Non reasoning openai models do not except this arg
 2025-08-20 19:07:28 -07:00
Debanjum
e504141c07 Fix to calculate usage from openai api streaming completion 
During streaming chunk.chunk contains usage data. This regression must
have appeared while tuning openai stream processors
 2025-08-20 19:07:28 -07:00
Debanjum
573c6a32e1 Fix to create chat with custom agents from obsidian (#1216) 
The function createNewConversation is never called with the agentSlug
specified so its always opening a new Conversation with the default Agent
 2025-08-20 19:07:16 -07:00
Debanjum
4728098cad Fix to set agent for new chat created from obsidian 
- Set the agent of the current conversation in the agent dropdown when a new conversation with a non-default agent is initialized. This was unset previously.
- Pass the current selected agent in the dropdown when creating new chat
- Correctly select the `khoj-header-agent-select' element
 2025-08-21 07:33:25 +05:30
Fh26697
a2a3eb8be6 Update chat_view.ts 
fixed Typo
 2025-08-20 17:01:47 +02:00
Fh26697
b3015f6837 Update chat_view.ts 
fixed Typo
 2025-08-20 16:58:08 +02:00
Fh26697
916534226a Chats are not using specified Agent 
The function createNewConversation is never called with the agentSlug specified so its always opening a new Conversation with the Base Agent
 2025-08-19 15:49:41 +02:00
Debanjum
fa143d45b9 Fix passing images to official openai models using the responses api 2025-08-17 16:30:43 -07:00
Debanjum
a494a766a4 Fix eval github workflow and show more logs to debug its startup 2025-08-15 16:26:37 -07:00
Debanjum
25e549d683 Show connection lost toast if disconnect while processing chat request 2025-08-15 16:04:26 -07:00
Debanjum
59bfaf9698 Fix to indicate ws disconnect on web app & save interrupted research 
- A regression had stopped indicating to user that the websocket
connection had broken. Now the interrupt has some visual indication.

- Websocket disconnects from client didn't trigger the partial
research to be saved. Now we use an interrupt signal to save partial
research before closing task.
 2025-08-15 16:04:26 -07:00
Debanjum
3eb8cce984 Retry if hit gemini rate limit. Return friendly message if retries fail 
Although we had handling in place for retrying after gemini suggested
backoff on hitting rate limits. The actual rate limit exception was
getting caught to render friendly message, so retry wasn't actually
getting triggered.

This change allows both
- Retry on hitting 429 rate limit exceptions
- Return friendly message if rate limit triggered retry eventually fails

Related:
- Changes to retry with gemini suggested backoff time in 0f953f9
 2025-08-15 16:04:25 -07:00
Debanjum
4274f58dbd Show more specific warning to llm on duplicate tool use during research 2025-08-15 16:02:32 -07:00
Debanjum
caf0b994e8 Fix handling failure to select default chat tools 
Issue: chosen_io variable was accessed before initialization when
ValueError was raise.

Fix: Set chosen_io to fallback values on failure to select default
chat tools
 2025-08-15 16:02:15 -07:00
Debanjum
7251b25c66 Handle null reference exceptions when rendering files context 2025-08-15 16:00:51 -07:00
Debanjum
20347e21c2 Reduce noisy indexing logs 2025-08-12 12:06:43 -07:00
Debanjum
bd82626084 Release Khoj version 2.0.0-beta.13 2025-08-11 22:29:06 -07:00
Debanjum
cbeefb7f94 Update researcher prompt to handle ambiguous queries. Clear stale text 
Make researcher handle ambiguous requests better by working with
reasonable assumptions (clearly told to user in response) instead of
burdering user with clarification requests.

Fix portions of the researcher prompt that had gone stale since moving
to tool use and making researcher more task (vs q&a) oriented
 2025-08-11 22:28:47 -07:00
Debanjum
0a6d87067d Fix to have researcher let the coder tool write code 
Previously the researcher was passing the whole code to execute in its
queries to the tool AI instead of asking it to write the code and
limiting its query to a natural language request (with required data).

The division of responsibility should help researcher just worry about
constructing a request with all the required details instead of also
worrying about writing correct code.
 2025-08-11 22:28:47 -07:00
Debanjum
0186403891 Limit retry to transient openai API errors. Return non-empty tool output 2025-08-11 21:53:21 -07:00
Debanjum
41f89cf7f3 Handle price, responses of models served via Groq 
Their tool call response may not strictly follow expected response
format. Let researcher handle incorrect arguments to code tool (i.e
triggers type error)
 2025-08-11 19:32:41 -07:00
Debanjum
b2d26088dc Use openai responses api to interact with official openai models 
What
- Get reasoning of openai reasoning models from responses api for sho
- Improves cache hits and reasoning reuse for iterative agents like
  research mode.

This should improve speed, quality, cost and transparency of using
openai reasoning models.

More cache hits and better reasoning as reasoning blocks are included
while model is researching (reasoning intersperse with tool calls)
when using the responses api.
 2025-08-09 14:03:24 -07:00
Debanjum
564adb24a7 Add support for GPT 5 model series 2025-08-09 14:03:13 -07:00
Debanjum
0e1615acc8 Fix grep files tool to work with line start, end anchors 
Previously line start, end anchors would just work if the whole file
started or ended with the regex pattern rather than matching by line.

Fix it to work like a standard grep tool and match by line start, end.
 2025-08-09 12:29:35 -07:00
Debanjum
a79025ee93 Limit max queries allowed per doc search tool call. Improve prompt 
Reduce usage of boolean operators like "hello OR bye OR see you" which
doesn't work and reduces search quality. They're trying to stuff the
search query with multiple different queries.
 2025-08-09 12:29:35 -07:00
Debanjum
a3bb7100b4 Speed up app development using a faster, modern toolchain (#1196) 
## Overview
Speed up app install and development using a faster, modern development
toolchain

## Details
### Major
- Use [uv](https://docs.astral.sh/uv/) for faster server install (vs
pip)
- Use [bun](https://bun.sh/) for faster web app install (vs yarn)
- Use [ruff](https://docs.astral.sh/ruff/) for faster formatting of
server code (vs black, isort)
- Fix devcontainer builds. See if uv and bun can speed up server and
client installs

### Minor
- Format web app with prettier and server with ruff. This is most of the
file changes in this PR.
- Simplify copying web app built files in pypi workflow to make it less
flaky.
 2025-08-09 12:27:20 -07:00
Debanjum
80cce7b439 Fix server, web app to reuse prebuilt deps on dev container setup 2025-08-01 23:36:13 -07:00
Debanjum
0a0b97446c Avoid `click' v8.2.2 server dependency as it breaks pypi validation 
Refer pallets/click issue 3024 for details
 2025-08-01 23:36:13 -07:00
Debanjum
f2bd07044e Speed up github workflows by not installing cuda server dependencies 
- CI runners don't have GPUs
- Pytorch related Nvidia cuda packages are not required for testing,
  evals or pre-commit checks.
- Avoiding these massive downloads should speed up workflow run.
 2025-08-01 23:35:08 -07:00
Debanjum
8ad38dfe11 Switch to Bun instead of Deno (or Yarn) for faster web app builds 2025-08-01 03:00:43 -07:00
Debanjum
b86430227c Dedupe and move dev dependencies out from web app production builds 2025-08-01 00:28:39 -07:00
Debanjum
791ebe3a97 Format web app code with prettier recommendations 
Too many of these had accumulated earlier from being ignored.
Changed to make build logs less noisy
 2025-08-01 00:28:39 -07:00
Debanjum
c8e07e86e4 Format server code with ruff recommendations 2025-08-01 00:28:17 -07:00
Debanjum
4a3ed9e5a4 Replace isort, black with ruff for faster linting, formatting 2025-08-01 00:01:34 -07:00
Debanjum
8700fb8937 Use UV, Deno for faster setup of development container 2025-08-01 00:01:34 -07:00
Debanjum
d2940de367 Use Deno for speed, package locks in dev setup, github workflows 
It's faster than yarn and comes with standard convenience utilities
 2025-08-01 00:01:34 -07:00
Debanjum
006b958071 Use UV to install server for speed, package locks in dev setup, workflows 
It's much faster than pip, includes dependency locks via uv.lock and
comes with standard convenience utilities (e.g pipx, venv replacement)
 2025-08-01 00:01:34 -07:00
Debanjum
e0f363d718 Use UV to manage python version, env on khoj computer 
- Use khoj username on khoj's computer
- Uv is much faster for builds
 2025-07-31 18:31:24 -07:00
Debanjum
0387b86a27 Use portable comparator to get flags used to call dev_setup.sh 2025-07-31 18:31:24 -07:00
Debanjum
c6670e815a Drop Server Side Indexer, Native Offline Chat, Old Migration Scripts (#1212) 
### Overview
Make server leaner to increase development speed. 
Remove old indexing code and the native offline chat which was hard to
maintain.

- The native offline chat module was written when the local ai model api
ecosystem wasn't mature. Now it is. Reuse that.
- Offline chat requires GPU for usable speeds. Decoupling offline chat
from Khoj server is the recommended way to go for practical inference
speeds (e.g Ollama on machine, Khoj in docker etc.)

### Details
- Drop old code to index files on server filesystem. Clean cli, init
paths.
- Drop native offline chat support with llama-cpp-python. 
  Use established local ai APIs like Llama.cpp Server, Ollama, vLLM etc.
- Drop old pre 1.0 khoj config migration scripts
- Update test setup to index test data after old indexing code removed.
 2025-07-31 20:26:08 -05:00
Debanjum
892d57314e Update test setup to index test data after old indexing code removed 
- Delete tests testing deprecated server side indexing flows
- Delete `Local(Plaintext|Org|Markdown|Pdf)Config' methods, files and
  references in tests
- Index test data via new helper method, `get_index_files'
  - It is modelled after the old `get_org_files' variants in main app
  - It passes the test data in required format to `configure_content'
    Allows maintaining the more realistic tests from before while
    using new indexing mechanism (rather than the deprecated server
    side indexing mechanism
 2025-07-31 18:25:32 -07:00
Debanjum
d9d24dd638 Drop old code to sync files on server filesystem. Clean cli, init paths 
This stale code was originally used to index files on server file
system directly by server. We currently push files to sync via API.

Server side syncing of remote content like Github and Notion is still
supported. But old, unused code for server side sync of files on
server fs is being cleaned out.

New --log-file cli args allows specifying where khoj server should
store logs on fs. This replaces the --config-file cli arg that was
only being used as a proxy for deciding where to store the log file.

- TODO
  - Tests are broken. They were relying on the server side content
    syncing for test setup
 2025-07-31 18:25:32 -07:00
Debanjum
b1f2737c9a Drop native offline chat support with llama-cpp-python 
It is recommended to chat with open-source models by running an
open-source server like Ollama, Llama.cpp on your GPU powered machine
or use a commercial provider of open-source models like DeepInfra or
OpenRouter.

These chat model serving options provide a mature Openai compatible
API that already works with Khoj.

Directly using offline chat models only worked reasonably with pip
install on a machine with GPU. Docker setup of khoj had trouble with
accessing GPU. And without GPU access offline chat is too slow.

Deprecating support for an offline chat provider directly from within
Khoj will reduce code complexity and increase developement velocity.
Offline models are subsumed to use existing Openai ai model provider.
 2025-07-31 18:25:32 -07:00
Debanjum
3f8cc71aca Drop old pre 1.0 khoj config migration scripts 
These were used when khoj was configured using khoj.yml file
 2025-07-31 18:25:32 -07:00
Debanjum
9096f628d0 Release Khoj version 2.0.0-beta.12 2025-07-31 18:13:17 -07:00
Debanjum
a6923fac76 Improve description of query arg to semantic, web search tool 
Clarify that the tool AI will perform a maximum of X sub-queries for
each query passed to it by the manager AI.

Avoids the manager AI from trying to directly pass a list of queries
to the search tool AI. It should just pass just a single query.
 2025-07-31 18:00:46 -07:00
Debanjum
2e13c9a007 Buffer thought chunks on server side for more performant ws streaming 
Send larger thought chunks to improve streaming efficiency and
reduce rendering load on web client.

This rendering load was most evident when using high throughput
models or low compute clients.

The server side message buffering should result in fewer re-renders,
faster streaming and lower compute load on client.

Related commit to buffer message content in fc99f8b37
 2025-07-31 18:00:46 -07:00
Debanjum
fba4ad27f7 Extract thought stream from reasoning_content of openai model providers 
Grok 3 mini at least sends thoughts in reasoning_content field of
streamed chunk delta. Extract model thoughts from that when available.
 2025-07-31 18:00:46 -07:00
Debanjum
b335f8cf79 Support grok 4 reasoning model 2025-07-31 18:00:46 -07:00
Debanjum
c0db9e4fca Use better, standard default temp, top_p for openai model providers 2025-07-31 18:00:46 -07:00
Debanjum
7ab24d875d Release Khoj version 2.0.0-beta.11 2025-07-31 10:25:42 -07:00
Debanjum
6290d744ea Make code tool write safe code to run in sandbox 
- Ask both manager and code gen AI to not run or write
  unsafe code for some safety improvement (over code exec in sandbox).
- Disallow custom agent prompts instructing unsafe code gen
 2025-07-31 00:11:50 -07:00
Debanjum
0f953f9ec8 Use Gemini suggested retry backoff if set. Improve gemini error handling 2025-07-30 18:16:16 -07:00
Debanjum
bbc14951b4 Redirect to a better error page on server error 2025-07-30 18:08:07 -07:00
Debanjum
6caa6f4008 Make async call to get agent files from async agent/conversation API 
This should avoid the sync_to_async errors thrown by django when
calling the /api/agent/conversation API endpoint
 2025-07-30 17:37:54 -07:00
Debanjum
b82d4fe68f Resolve Pydantic deprecation warnings (#1211) 
## PR Summary
This PR resolves the deprecation warnings of the Pydantic library, which
you can find in the [CI
logs](https://github.com/khoj-ai/khoj/actions/runs/16528997676/job/46749452047#step:9:142):
```python
PydanticDeprecatedSince20: The `copy` method is deprecated; use `model_copy` instead. See the docstring of `BaseModel.copy` for details about how to handle `include` and `exclude`. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.11/migration/
```
 2025-07-25 19:50:57 -05:00
Emmanuel Ferdman
655a1b38f2 Resolve Pydantic deprecation warnings 
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
 2025-07-25 16:55:00 -07:00
Debanjum
f5d12b7546 Bump desktop app and documentation dependencies 2025-07-25 13:37:45 -05:00
Debanjum
f8924f2521 Avoid duplicate chat turn save if chat cancelled during final response 
Save to conversation in normal flow should only be done if
interrupt wasn't triggered.

Saving conversations on interrupt is handled completely by the
disconnect monitor since the improvements to interrupt.

This abort is handled correctly for steps before final response. But
not if interrupt occurs while final response is being sent. This
changes checks for cancellation after final response send attempt and
avoids duplicate chat turn save.
 2025-07-25 13:28:13 -05:00
Debanjum
bd9f091a71 Show thoughts of more llm models served via openai compatible api 
- Extract llm thoughts from more openai compatible ai api providers
  like llama.cpp server vllm and litellm.
  - Try structured thought extraction by default
  - Try in-stream thought extraction for specific model families like
    qwen and deepseek.
- Show thoughts with tool use. For intermediate steps like research
  mode from openai compatible models

Some consensus on thought in model response is being reached with
using deepseek style thoughts in structured response (via
"reasoning_content" field)  or qwen style thoughts in main
response (i.e <think></think> tags).

Default to try deepseek style structured thought extraction. So the
previous default stream processor isn't required.
 2025-07-25 13:28:13 -05:00
Debanjum
624d6227ca Expand to enable deep think for more qwen style models like smollm3 2025-07-25 13:28:13 -05:00
Debanjum
c401bb9591 Stricty enforce tool call schema for llm served via openai compat api 
This is required by llama.cpp server and is recommended in general for
openai compatible models
 2025-07-25 13:28:13 -05:00
Debanjum
03c4f614dd Handle tool call requests with openai completion in non stream mode 2025-07-25 13:28:13 -05:00
Debanjum
70cfaf72e9 Only send start llm response chat event once, after thoughts streamed 
A previous regression resulted in the start llm response event being
sent with every (non-thought) message chunk. It should only be sent
once after thoughts and before first normal message chunk is streamed.

Regression probably introduced with changes to stream thoughts.

This should fix the chat streaming latency logs.
 2025-07-25 13:28:13 -05:00
Debanjum
15c6118142 Store event delimiter in chat event enum for reuse 2025-07-25 13:28:13 -05:00
Debanjum
fc99f8b37e Buffer message chunks on server side for more performant ws streaming 
Send larger message chunks to improve streaming efficiency and
reduce rendering load on web client.

This rendering load was most evident when using high throughput
models, low compute clients and message with images. As message
content was rerendered on every token sent to the web app.

The server side message buffering should result in fewer re-renders
and lower compute load on client.
 2025-07-25 13:28:13 -05:00
Debanjum
bf9a9c7283 Disable per user websocket connection limits in anon or debug mode 
This rate limiting is only relevant in production scenarios.
 2025-07-25 12:19:15 -05:00
Debanjum
48e21d9f0f Release Khoj version 2.0.0-beta.10 2025-07-19 21:32:14 -05:00
Debanjum
e57acf617a Convert websocket rate limiter to async method 
Fixes calling websocket rate limiter from async chat_ws method.

Not sure why the issue did not trigger in local setups. Maybe has to
do with gunicorn vs uvicorn / multi-workers setup in prod vs local.
 2025-07-19 21:15:51 -05:00
Debanjum
76a1b0b686 Release Khoj version 2.0.0-beta.9 2025-07-19 20:20:50 -05:00
Debanjum
43d7e65a49 Limit chat message interrupt queue size to limit performance impact 2025-07-19 20:16:53 -05:00
Debanjum
749160e38d Validate websocket origin before establishing connection 2025-07-19 20:07:21 -05:00
Debanjum
69a7d332fc Limit number of new websocket connections allowed per user 2025-07-19 20:04:36 -05:00
Debanjum
76ddf8645c Improve rate limit and interrupt messages for user, admin 2025-07-19 19:13:51 -05:00
Debanjum
de7668daec Add websocket chat api to ease bi-directional communication (#1207) 
- Add a websocket api endpoint for chat. Reuse most of the existing chat
logic.
- Communicate from web app using the websocket chat api endpoint.
- Pass interrupt messages using websocket to guide research, operator
trajectory
Previously we were using the abort and send new POST /api/chat
mechanism.
This didn't scale well to multi-worker setups as a different worker
could pick up the new interrupt message request.
  Using websocket to send messages in the middle of long running tasks
  should work more naturally.
 2025-07-17 18:06:43 -07:00
Debanjum
b90e2367d5 Fix interrupt UX and research when using websocket via web app 2025-07-17 17:09:21 -07:00
Debanjum
0ecd5f497d Show more informative title for semantic search train of thought 2025-07-17 17:09:21 -07:00
Debanjum
7b7b1830b7 Make callers only share new messages to append to chat logs 
- Chat history is retrieved and updated with new messages just before
  write. This is to reduce chance of message loss due to conflicting
  writes making last to save to conversation win conflict.
- This was problematic artifact of old code. Removing it should reduce
  conflict surface area.
- Interrupts and live chat could hit this issue due to different reasons
 2025-07-17 17:09:18 -07:00
Debanjum
eaed0c839e Use websocket chat api endpoint to communicate from web app 
- Use websocket library to handle setup, reconnection from web app
Use react-use-websocket library to handle websocket connection and
reconnection logic. Previously connection wasn't re-established on
disconnects.

- Send interrupt messages with ws to update research, operator trajectory

Previously we were using the abort and send new POST /api/chat
mechanism.

But now we can use the websocket's bi-directional messaging capability
to send users messages in the middle of a research, operator run.

This change should
1. Allow for a faster, more interactive interruption to shift the
research direction without breaking the conversation flow. As
previously we were using the DB to communicate interrupts across
workers, this would take time and feel sluggish on the UX.

2. Be a more robust interrupt mechanism that'll work in multi worker
setups. As same worker is interacted with to send interrupt messages
instead of potentially new worker receiving the POST /api/chat with
the interrupt user message.

On the server we're using an asyncio Queue to pass messages down from
websocket api to researcher via event generator. This can be extended
to pass to other iterative agents like operator.
 2025-07-17 17:06:55 -07:00
Debanjum
9f0eff6541 Handle passing interrupt messages from api to chat actors on server 2025-07-17 17:06:55 -07:00
Debanjum
38dd85c91f Add websocket chat api endpoint to ease bi-directional communication 2025-07-17 17:06:55 -07:00
Debanjum
99ed796c00 Release Khoj version 2.0.0-beta.8 2025-07-15 16:42:44 -07:00
Debanjum
0a05a5709e Use agent chat model to generate code instead of default chat model 
This is consistent with chat model preference order for other tools
 2025-07-15 16:22:29 -07:00
Debanjum
238bd66c42 Fix to map user tool names to equivalent tool sets for research mode 
Fix using research tool names instead of slash command tool names
(exposed to user) in research mode conversation history construction.

Map agent input tools to relevant research tools. Previously
using agents with a limited set of tools in research mode reduces
tools available to agent in research mode.

Fix checks to skip tools if not configured.
 2025-07-15 16:22:29 -07:00
Debanjum
76ed97d066 Set friendly name for auto loaded chat models during first run 
The chat model friendly name field was introduced in a8c47a70f. But
we weren't setting the friendly name for ollama models, which get
automatically loaded on first run.

This broke setting chat model options, server admin settings and
creating new chat pages (at least) as they display the chat model's
friendly name.

This change ensures the friendly name for auto loaded chat models is
set to resolve these issues. We also add a null ref check to web app
model selector as an additional safeguard to prevent new chat page
crash due to missing friendly name going forward.

Resolves #1208
 2025-07-15 14:27:04 -07:00
Debanjum
0a06f5b41a Release Khoj version 2.0.0-beta.7 2025-07-11 00:04:56 -07:00
Debanjum
d42176fa7e Drop tool call, result without tool id on call to Anthropic, Openai APIs 2025-07-11 00:00:05 -07:00
Debanjum
d27aac7f13 Suppress non-actionable pdf indexing warning from logs 2025-07-11 00:00:05 -07:00
Debanjum
05176cd62b Log dropping messages with invalid content as warnings, not errors 
They are expected when conversation got interrupted.
 2025-07-11 00:00:05 -07:00
Debanjum
b2952236c4 Log conversation id to help troubleshoot errors faster 2025-07-10 23:56:42 -07:00
Debanjum
25db59e49c Fix to return openai formatted messages in the correct order 
We'd reversed the formatting of openai messages to drop invalid
messages without affecting the other messages being appended . But we
need to reverse the final formatted list to return in the right order.
 2025-07-10 23:56:22 -07:00
Debanjum
c8ec29551f Drop invalid messages in reverse order to continue interrupted chats 
Previously
- message with invalid content were getting dropped in normal order
  which would change the item index being iterated for gemini and
  anthropic models
- messages with empty content weren't getting dropped for openai
  compatible api models. While openai api is resilient to this, it's
  better to drop these invalid messages as other openai compatible
  APIs may not handle this.

We see messages with empty or no content when chat gets interrupted
due to disconnections, interrupt messages or explicit aborts by user.

This changes should now drop invalid messages and not mess formatting
of the other messages in a conversation. It should allow continuing
interrupted conversations with any ai model.
 2025-07-10 22:39:52 -07:00
Debanjum
f1a3ddf2ca Release Khoj version 2.0.0-beta.6 2025-07-10 13:41:06 -07:00
Debanjum
7b637d3432 Use document style ux when print conversations to pdf 
Inspired by my previous turnstyle ux explorations.

But basically user message becomes section title and khoj message
becomes section body with the timestamp being used a section title,
body divider.
 2025-07-10 13:27:04 -07:00
Debanjum
c28e90f388 Revert to use standard 1.0 temperature for gemini models 
Using temp of 1.2 didn't help eliminate the repetition loops the
gemini models go into sometimes.
 2025-07-09 18:22:05 -07:00
Debanjum
b763dbfb2b Timeout web search and webpage read requests to providers 2025-07-09 18:12:07 -07:00
Debanjum
1988a8d023 Fix to delete agent by slug in DB via API 2025-07-09 18:12:07 -07:00
Debanjum
69336565b1 Do not show research mode tools as slash commands options on clients 
These are tools meant for the research agent, not for users to use.
 2025-07-09 18:12:07 -07:00
Debanjum
cc6da4c440 Drop unsupported additionalProperties field from gemini tool definitions 2025-07-09 18:06:40 -07:00
Debanjum
3141035f48 Handle unexpected chunks streamed from Openai (compatible) APIs 2025-07-09 17:54:42 -07:00
Debanjum
a601cca79b Handle cases where no organic online search results found 
Previous organic results enumerator only handled the scenario where
organic key wasn't present in online search results.

It did not handle the case where there were no organic online search
results.
 2025-07-09 00:25:10 -07:00
Debanjum
f2b86aa7c8 Release Khoj version 2.0.0-beta.5 2025-07-08 23:45:29 -07:00
Debanjum
0f0cfba624 Ignore vscode settings.json from pre-commit json check 
Vscode settings.json follows jsonc (json with comments) format
 2025-07-08 23:27:10 -07:00
Debanjum
f0513cbbb1 Fix to run new automation api tests in ci 2025-07-08 23:27:09 -07:00
Debanjum
c144aa9c90 Handle automation calling url of both url and string type 
Calling url can be of url type in production but locally it is of
string type. Unclear why. But this change should mitigate the issue
for now.
 2025-07-08 21:10:54 -07:00
Debanjum
fad6a638bd Release Khoj version 2.0.0-beta.4 2025-07-08 19:34:52 -07:00
Debanjum
8d9e75f580 Fix automation url parsing, response handling. Test automations api 
- Methods calling send_message_to_model_wrapper_sync handn't been
  update to handle the function returning new ResponseWithThought
- Store, load request.url to DB as, from string to avoid serialization
  issues
 2025-07-08 17:40:19 -07:00
Debanjum
254207b010 Make chats print friendly to share via print to PDF etc. from browser 
Add print specific styling to hide side panels and chat input footers.
Add heading with khoj logo, conversation title, agent and date.
 2025-07-08 12:20:19 -07:00
Debanjum
8fb38d9e1e Make vscode pylint only analyse khoj server directory for efficiency 2025-07-08 12:20:19 -07:00
Debanjum
9a215141f0 Release Khoj version 2.0.0-beta.3 2025-07-06 12:50:27 -07:00
Debanjum
da9a78e79b Make URI field optional for now to handle previously saved documents 
For files not synced after the previous release, context uri is unset.
This results in failure to save chat messages that retrieve documents
as the uri field cannot be unset so pre save validation fails.

We'd use a db migration to handle this but this is a quick mitigation
for now.
 2025-07-06 12:49:07 -07:00
Debanjum
bc6bbb4c96 Release Khoj version 2.0.0-beta.2 2025-07-06 12:10:32 -07:00
Debanjum
4c33d1a526 Fallback to file based URI when document context URI is unset 
For files not synced after the previous release, context uri is unset.
This results in failure to save chat messages that retrieve documents
as the uri field cannot be unset so pre save validation fails
 2025-07-06 12:06:02 -07:00
Debanjum
9dc146bb08 Bump rapidocr dependency version 2025-07-06 12:06:02 -07:00
Debanjum
b27ba1d24b Early init chat_history in chat api to avoid unbound in edge case 
Monitor disconnect can trigger earlier than chat history is
initialized. This can cause unbound chat history exception.
 2025-07-06 12:06:02 -07:00
Debanjum
8cd2a1a961 Release Khoj version 2.0.0-beta.1 2025-07-06 10:39:56 -07:00
Debanjum
6bda8dc20b Fix file upload size limit client test after max upload bump 2025-07-06 10:24:44 -07:00
Debanjum
531ae80212 Allow 50mb knowledge base size on free tier 2025-07-06 09:56:29 -07:00
Debanjum
58f44ad43b Make release/1.x a privileged branch to run workflows, create releases 
It'll work similar to the master branch but with pre-1x and latest-1x
tagged series of docker images.

This should ease deployment changes from 1.x vs 2.x series
 2025-07-06 09:28:16 -07:00
Debanjum
2daf396cbb Bump pre-release version in SemVer schema via bump_version script 2025-07-06 09:28:16 -07:00
Debanjum
afa810e552 Retry api calls to gemini on network read error 2025-07-06 09:27:54 -07:00
Debanjum
2ec39d295d Add Deeplinks to Improve Context for Document Retrieval (#1206) 
## Overview
Show deep link URI and raw document context to provide deeper, richer
context to Khoj. This should allow it better combine semantic search
with other new document retrieval tools like line range based file
viewer and regex tools added in #1205

## Details
- Attach line number based deeplinks to each indexed document entry
Document URI follows URL fragment based schema of form
`file:///path/to/file.txt#line=123`
- Show raw indexed document entries with deep links to LLM when it uses
the semantic search tool
- Reduce structural changes to raw org-mode entries for easier deep
linking.
 2025-07-03 20:05:04 -07:00
Debanjum
5010623a0a Deep link to markdown entries by line number in uri 
Use url fragment schema for deep link URIs, borrowing from URL/PDF
schemas. E.g file:///path/to/file.txt#line=<line_no>&#page=<page_no>

Compute line number during (recursive) markdown entry chunking.

Test line number in URI maps to line number of chunk in actual md file.

This deeplink URI with line number is passed to llm as context to
better combine with line range based view file tool.

Grep tool already passed matching line number. This change passes
line number in URIs of markdown entries matched by the semantic search
tool.
 2025-07-03 19:27:57 -07:00
Debanjum
dcfa4288c4 Deep link to org-mode entries. Deep link by line number in uri 
Use url fragment schema for deep link URIs, borrowing from URL/PDF
schemas. E.g file:///path/to/file.txt#line=<line_no>&#page=<page_no>

Compute line number during (recursive) org-mode entry chunking.

Thoroughly test line number in URI maps to line number of chunk in
actual org mode file.

This deeplink URI with line number is passed to llm as context to
better combine with line range based view file tool.

Grep tool already passed matching line number. This change passes
line number in URIs of org entries matched by the semantic search tool
 2025-07-03 17:38:34 -07:00
Debanjum
e90ab5341a Add context uri field to deeplink line number in original doc 2025-07-03 17:38:34 -07:00
Debanjum
820b4523fd Show raw rather than compiled entry to llm and users 
Only embedding models see, operate on compiled text.

LLMs should see raw entry to improve combining it with other document
traversal tools for better regex and line matching.

Users see raw entry for better matching with their actual notes.
 2025-07-03 17:38:34 -07:00
Debanjum
5c4d41d300 Reduce structural changes to indexed raw org mode entries 
Reduce structural changes to raw entry allows better deep-linking and
re-annotation. Currently done via line number in new uri field.

Only add properties drawer to raw entry if entry has properties
Previously line and source properties were inserted into raw entries.
This isn't done anymore. Line, source are deprecated for use in khoj.el.
 2025-07-03 17:38:31 -07:00
sabaimran
870d9d851a Only handle Stripe webhooks meant for the KHOJ_CLOUD product 2025-07-03 17:02:49 -07:00
Debanjum
fe44cd3c59 Upgrade Retrieval from KB in Research Mode. Use Function Calling for Tool Use (#1205) 
## Why
Move to function calling paradigm to give models tool call -> tool
result in formats they're fine-tuned to understand. Previously we were
giving them results in our specific format (as function calling paradigm
wasn't well-established yet).

And improve prompt cache hits by caching tool definitions.

This is a **breaking change**. AI Models and APIs that do not support
function calling will not work with Khoj in research mode. Function
calling is supported by:
- Standard commercial AI Models and APIs like Anthropic, Gemini, OpenAI,
OpenRouter
- Standard open-source AI APIs like llama.cpp server, Ollama
- Standard open source models like Qwen, DeepSeek, Gemma, Llama, Mistral

## What
### Use Function Calling for Tool Use
- Add Function Calling support to Anthropic, Gemini, OpenAI AI Model
APIs
- Move Existing Research Mode Tools to Use Function Calling

### Get More Comprehensive Results from your Knowledge Base (KB)
- Give Research Agent better Document Retrieval Tools
  - Add grep files tool to enable researcher to find documents by regex
  - Add list files tool to enable researcher to find documents by path
  - Add file viewer tool to enable researcher to read documents

### Miscellaneous
- Improve Research Prompt, Truncation, Retry and Caching
- Show reasoning model thoughts in Khoj train of thought for
intermediate steps as well
 2025-07-03 00:14:07 -07:00
Debanjum
f343a92b1d Give research tools better, consistent names for balanced usage 2025-07-02 23:32:44 -07:00
Debanjum
aa081913bf Improve truncation with tool use and Anthropic caching 
- Cache last anthropic message. Given research mode now uses function
  calling paradigm and not the old research mode structure.
- Cache tool definitions passed to anthropic models
- Stop dropping first message if by assistant as seems like Anthropic
  API doesn't complain about it any more.

- Drop tool result when tool call is truncated as invalid state
- Do not truncate tool use message content, just drop the whole tool
  use message.

  AI model APIs need tool use assistant message content in specific
  form (e.g with thinking etc.). So dropping content items breaks
  expected tool use message content format.

Handle tool use scenarios where iteration query isn't set for retry
 2025-07-02 23:32:44 -07:00
Debanjum
786b06bb3f Handle failed llm calls, message idempotency to improve retry success 
- Deepcopy messages before formatting message for Anthropic to allow
  idempotency so retry on failure behaves as expected
- Handle failed calls to pick next tools to pass failure warning and
  continue next research iteration. Previously if API call to pick
  next failed, the research run would crash
- Add null response check for when Gemini models fail to respond
 2025-07-02 23:32:30 -07:00
Debanjum
30878a2fed Show thoughts and text response in thoughts on anthropic tool use 
Previously if anthropic models were using tools, the models text
response accompanying the tool use wouldn't be shown as they were
overwritten in aggregated response with the tool call json.

This changes appends the text response to the thoughts portion on tool
use to still show model's thinking. Thinking and text response are
delineated by italics vs normal text for such cases.
 2025-07-02 20:48:24 -07:00
Debanjum
c2ab75efef Track, reuse raw model response for multi-turn conversations 
This should avoid the need to reformat the Khoj standardized tool call
for cache hits and satisfying ai model api requirements.

Previously multi-turn tool use calls to anthropic reasoning models
would fail as needed their thoughts to be passed back. Other AI model
providers can have other requirements.

Passing back the raw response as is should satisfy the default case.

Tracking raw response should make it easy to apply any formatting
required before sending previous response back, if any ai model
provider requires that.

Details
---
- Raw response content is passed back in ResponseWithThoughts.
- Research iteration stores this and puts it into model response
  ChatMessageModel when constructing iteration history when it is
  present.
  Fallback to using parsed tool call when raw response isn't present.
- No need to format tool call messages for anthropic models as we're
  passing the raw response as is.
 2025-07-02 20:48:24 -07:00
Debanjum
7cd496ac19 Frame research prompt as accomplish task instead of answer question 
Researcher is expanding into accomplish task behavior, especially with
tool use from the previous collect information to answer user query
behavior.

Update the researcher's system prompt to reflect the new objective better.
Encourage model to not stop working on task until achieve objective
 2025-07-02 20:48:24 -07:00
Debanjum
4e67ba4d6c Support seeing lines around regex match with grep files tool 
Let research agent see lines surrounding regex matched lines when
using grep files tool to improve document retrieval quality
 2025-07-02 20:48:24 -07:00
Debanjum
d81fb08366 Use case insensitive regex matching with grep files tool 2025-07-02 20:48:24 -07:00
Debanjum
9c38326608 Add grep files tool to enable researcher to find documents by regex 
Earlier khoj could technically only answer questions existential
questions, i.e question that would terminate once any relevant note to
answer that question was found.

This change enables khoj to answer universal questions, i.e questions
that require searching through all notes or finding all instances.

It enables more thorough retrieval from user's knowledge base by
combining semantic search, regex search, view and list files tools.

For more development details including motivation, see live coding
session 1.1 at https://www.youtube.com/live/-2s_qi4hd2k
 2025-07-02 20:48:24 -07:00
Debanjum
59f5648dbd Add list files tool to enable researcher to find documents by path 
Allow getting a map of user's knowledge base under specified path.

This enables more thorough retrieval from user's knowledge base by
combining search, view and list files tools.
 2025-07-02 20:48:24 -07:00
Debanjum
2f9f608cff Add file viewer tool to enable researcher to read documents 
Allow reading whole file contents or content in specified line range
in user's knowledge base. This allows for more deterministic
traversal.
 2025-07-02 20:48:24 -07:00
Debanjum
721c55a37b Rename ResponseWithThought response field to text for better naming 2025-07-02 20:48:24 -07:00
Debanjum
490f0a435d Pass research tools directly with their varied args for flexibility 
Why
---
Previously researcher had a uniform response schema to pick next tool,
scratchpad, query and tool. This didn't allow choosing different
arguments for the different tools being called. And the tool call,
result format passed by khoj was custom and static across all LLMs.

Passing the tools and their schemas directly to llm when picking next
tool allows passing multiple, tool specific arguments for llm to
select. For example, model can choose webpage urls to read or image
gen aspect ratio (apart from tool query) to pass to the specific tool.

Using the LLM tool calling paradigm allows model to see tool call,
tool result in a format that it understands best.

Using standard tool calling paradigm also allows for incorporating
community builts tools more easily via MCP servers, clients tools,
native llm api tools etc.

What
---
- Return ResponseWithThought from completion_with_backoff ai model
  provider methods
- Show reasoning model thoughts in research mode train of thought.
  For non-reasoning models do not show researcher train of thought.
  As non-reasoning models don't (by default) think before selecing
  tool. Showing tool call is lame and resembles tool's action shown in
  next step.

- Store tool calls in standardized format.
- Specify tool schemas in tool for research llm definitions as well.
- Transform tool calls, tool results to standardized form for use
  within khoj. Manage the following tool call, result transformations:
  - Model provider tool_call -> standardized tool call
  - Standardized tool call, result -> model specific tool call, result

- Make researcher choose webpages urls to read as well for the webpage
  tool. Previously it would just decide the query but let the webpage
  reader infer the query url(s). But researcher has better context on
  which webpages it wants to have read to answer their query.

  This should eliminate the webpage reader deciding urls to read step
  and speed up webpage read tool use.

Handle unset response thoughts. Useful when retry on failed request

Previously resulted in unbound local variable response_thoughts error
 2025-07-02 20:48:23 -07:00
Debanjum
80522e370e Make researcher pick next tool using model function calling feature 
The pick next tool requests next tool to call to model in function
calling / tool use format.
 2025-07-02 19:10:02 -07:00
Debanjum
b888d5e65e Add function calling support to Anthropic, Gemini and OpenAI models 
Previously these models could use response schema but not tools use
capabilities provided by these AI model APIs.

This change allows chat actors to use the function calling feature to
specify which tools the LLM by these providers can call.

This should help simplify tool definition and structure context in
forms that these LLMs natively understand.
(i.e in tool_call - tool_result ~chatml format).
 2025-07-02 19:10:02 -07:00
Debanjum
9607f2e87c Release Khoj version 1.42.8 2025-07-02 19:07:51 -07:00
Debanjum
f4fc76645c Upgrade electron package used by desktop app 2025-07-02 18:50:18 -07:00
Debanjum
96fb9bd87c Tune temperature and top_p to reduce gemini model repetition 
Gemini models, especially flash models, seems to have a tendency to go
into long, repetitive output tokens loop. Unsure why.

Tune temp, top_p as gemini api doesn't seem to allow setting frequency
or presence penalty, at least for reasoning models. Those would have
been a more direct mechanism to avoid model getting stuck in a loop.
 2025-07-02 18:42:32 -07:00
Debanjum
9774bb012e Update agent knowledge base and configuration atomically 
This should help prevent partial updates to agent. Especially useful
for agent's with large knowledge bases being updated. Failing the call
should raise an exception. This will allow your to retry save instead
of losing your previous agent changes or saving only partial.
 2025-07-02 18:01:18 -07:00
Debanjum
e6cc9b1182 Test update agents with large knowledge bases 2025-07-02 18:01:18 -07:00
Debanjum
5fe2ea8a55 Run safety check only when agent persona updated on agent edits 
Running safety check isn't required if the agent persona wasn't
updated this time around as it would have passed safety check
previously.

This should speed up editing agents when agent persona isn't updated.
 2025-07-02 18:01:18 -07:00
Debanjum
a8c47a70f7 Show friendly name for available ai models on clients when set 2025-07-01 16:59:13 -07:00
Debanjum
487826bc32 Release Khoj version 1.42.7 2025-06-27 18:21:18 -07:00
Debanjum
29e5d7ef08 Improve support for new Deepseek R1 model over Openai compatible api 
Parse thinking out from <think>..</think> tags in chat response
Handle merging structured message content, not just str, for deepseek.
 2025-06-27 18:17:35 -07:00
Debanjum
a33580d560 Enable cache, proxy to improve firecrawl webpage scrape speed, success 2025-06-27 16:35:25 -07:00
Debanjum
1566e3c74d Ease bulk (de-)selecting of files to add/remove to agent knowledge base 
Add select all, deselect all buttons to select all (filtered) files to
add, remove from an agent's knowledge base.
 2025-06-27 15:19:50 -07:00
Debanjum
3bb4e63f3e Add ability to set default chat model via env var in docker-compose.yml 2025-06-27 15:19:50 -07:00
Debanjum
dd89dd3fc8 Bump web, documentation and desktop app package dependencies 2025-06-27 15:19:50 -07:00
Peter Gaultney
9f3ceba541 Allow setting embedded postgres db directory with PGSERVER_DATA_DIR env var (#1202) 
It seems to me that it would be useful to be able to be explicit about
where the embedded database should live - as well as where it _does_
live (via the info log), when not specifying.
 2025-06-28 03:21:23 +05:30
Debanjum
d37113850c Let reasoning gemini models dynamically set their thinking budget 
All gemini 2.5 series models support dynamic thinking budgets by
setting thinking_budget to -1.
 2025-06-27 13:13:24 -07:00
Debanjum
ba059ad8b0 Deduplicate passing chat history to extract question only in prompt 
Extract questions has chat history in prompt and in actual chat history.

Only pass in prompt for now. Later update prompts to pass chat history
in chat messages list for better truncation flexibility.
 2025-06-24 02:49:29 -07:00
Debanjum
170a8036fe Fix 2 document retrieval bugs to not drop valid search results 
1. Due to the interaction of two changes:
  - dedupe by corpus_id, where corpus_id tracks logical content blocks
  like files, org/md headings.
  - return compiled, not logical blocks, where compiled track smaller
  content chunks that fit within search model, llm context windows.

  When combined they showed only 1 hit compiled chunk per logical
  block. Even if multiple chunks match within a logical content block.

  Fix is to either dedupe by compiled text or to return deduped
  logical content blocks (by corpus_id) corresponding to matched
  compiled chunks. This commit fixes it by the first method.

2. Due to inferred query, search results zip which resulted in a
   single search result being returned per query!
   This silently cut down matching search results and went undetected.
 2025-06-24 02:47:07 -07:00
Debanjum
73c384b052 Reduce chat history spacing to reduce wasted space b/w chat input box 
The tailwing theme spacing of the scroll area surrounding chat history
on large screens was what was causing the large gap between chat input
box and chat history on some screen layouts.

This change reduces the spacing to a more acceptable level.
 2025-06-24 02:46:46 -07:00
Debanjum
ca9109455b Retry on intermitted image generation failure for resilient generation 2025-06-24 02:46:46 -07:00
Debanjum
4448ab665c Improve google image generation configuration 2025-06-24 02:46:46 -07:00
Debanjum
dc202e4441 Release Khoj version 1.42.6 2025-06-20 15:00:22 -07:00
Debanjum
623c8b65f1 Set failed response message when a research iteration fails. 
Previously summarizedResult would be unset when a tool call failed.

This caused research to fail due to ChatMessageModel failures when
constructing tool chat histories and would have caused similar errors
in other constructed chat histories.

Putting a failed iteration message in the summary prevents that while
letting the research agent continue its research.
 2025-06-20 14:13:50 -07:00
Debanjum
b85c646611 Make organic web search result text snippet field optional 
All web search providers, like Jina/Searxng?, do not return a text
snippet. Making snippet optional allows processing search results by
these web search providers, without hitting validation errors.
 2025-06-20 13:47:08 -07:00
Debanjum
22d71cab44 Log ChatMessageModel validation errors during conversation save 2025-06-19 16:48:11 -07:00
Debanjum
494e7b3856 Update gemini 2.5 to stable model pricing from preview pricing 2025-06-19 16:48:11 -07:00
Debanjum
029bd3be56 Handle breaking change in write file to e2b code sandbox 
For some reason the function signature, kwargs are broken. Removing
usage of keyword args resolves the file upload to sandbox error.
 2025-06-19 16:48:11 -07:00
Debanjum
b18b7b2e33 Handle unset response thoughts. Useful when retry on failed request 
Previously resulted in unbound local variable response_thoughts error
 2025-06-19 16:48:06 -07:00
Debanjum
906ff46e6c Handle research iterations where document search returns no results 2025-06-19 16:47:08 -07:00
Debanjum
aa7b23c125 Handle rendering document references with no compiled text on web app 2025-06-17 15:47:58 -07:00
Debanjum
4ca247f0bc Always append random suffix to shared conversations urls 2025-06-17 15:47:58 -07:00
Debanjum
68b7057a76 Share https url unless explicitly disabled or on localhost 2025-06-17 15:47:58 -07:00
Debanjum
bdda03b0bf Git ignore obsidian config directories 2025-06-16 12:01:19 -07:00
Debanjum
e635b8e3b9 Handle gemini chat response completion chunk when streaming 2025-06-13 18:36:53 -07:00
Debanjum
963ebc8875 Pass query params to doc search function before user, chat history 
Makes document search arg ordering more consistent with other tools
like online search, run code etc.
 2025-06-13 13:29:30 -07:00
Debanjum
9673f8beba Release Khoj version 1.42.5 2025-06-11 13:36:46 -07:00
Debanjum
e87be4edf4 Pin python version used by github workflow to publish to pypi 
Avoids having to update python path to write web app static build
files to everytime patch version of python is updated
 2025-06-11 13:30:15 -07:00
Debanjum
eaae1cf74e Fix rendering thoughts of Gemini reasoning models 
Previously there was duplication of thought in message to user and in
the train of thought. This should be resolved now
 2025-06-11 13:09:38 -07:00
Debanjum
4946ea1668 Fix to save organic results to conversation context in DB 
This bug was introduced in 05d4e19cb, version 1.42.2, during migration
to save deeply typed ChatMessageModel. As the ChatMessageModel did
not use the right field name for organic results (since the start).

Previously it did not matter as it was storing to DB irrespective but
now the mapping of dictionary to ChatMessageModel drops that field
before save to conversation in DB.

This was resulting in organic context being lost on page reload and
only being shown on first response.
 2025-06-11 12:52:42 -07:00
Debanjum
30ced1d86c Log non schema adhering chat message before save to DB 2025-06-11 12:52:42 -07:00
Debanjum
71763684a9 Explicitly drop stream_options if not streaming openai chat response 
Not sure why but it some cases when interacting with o3 (which needs
non-streaming) the stream_options seems to be set.

Cannot reproduce but hopefully dropping the stream_options explicitly
should resolve this issue.

Related 985a98214
 2025-06-11 12:52:42 -07:00
Debanjum
65644f78b0 Set lower max output tokens for non reasoning Gemini models 
While reasoning models support longer output tokens. Non reasoning
models do not. Use a lower max output tokens for them
 2025-06-11 11:12:24 -07:00
Debanjum
71221533c8 Release Khoj version 1.42.4 2025-06-10 23:49:30 -07:00
Debanjum
985a982148 Update openai package to stream response by non-reasoning models 
Older package (like 1.84.0) seem to always pass reasoning_effort
argument to openai api, which now seems to be throwing unexpected
request argument error when used with non-reasoning models (like
4o-mini).
 2025-06-10 23:49:04 -07:00
Debanjum
9b767438e2 Update model pricing, default models, context and version metadata 2025-06-10 23:49:04 -07:00
Debanjum
753972997f Enable non-streaming response via openai api to support o3 models 2025-06-10 23:49:04 -07:00
Debanjum
5110a06085 Fix GET agents API to return agent specific chat model 
There had been a regression that made all agents display the default
chat model instead of the actual chat model associated with the agent.

This change resolves that issue by prioritizing agent specific chat
model from DB (over user or server chat model).
 2025-06-10 15:29:46 -07:00
Debanjum
0cd709caf4 Release Khoj version 1.42.3 2025-06-10 10:20:44 -07:00
Debanjum
313f648bd7 Compile ai message content into single string when using DeepInfra 
DeepInfra only accepts assistant message.content of string type
 2025-06-10 01:58:43 -07:00
Debanjum
9e73309d01 Add no think tag for qwen models msgs over api when no deepthought 2025-06-10 01:58:43 -07:00
Debanjum
64886cd0dd Fix storing code results on server and rendering them on web app 
- Fix code context data type for validation on server. This would
  prevent the chat message from being written to history
- Handle null code results on web app
 2025-06-09 23:46:12 -07:00
Debanjum
b1a6e53d77 Fix populating chat message history to continue interrupted research 
We now pass deeply typed chat messages throughout the application to
construct tool specific chat history views since 05d4e19cb.

This ChatMessageModel didn't allow intent.query to be unset. But
interrupted research iteration history can have unset query. This
changes allows makes intent.query optional.

It also uses message by user entry to populate user message in tool
chat history views. Using query from khoj intent was an earlier
shortcut used to not have to deal with message by user. But that
doesn't scale to current scenario where turns are not always required
to have a single user, assistant message pair.

Specifically a chat history can now contain multiple user messages
followed by a single khoj message. The new change constructs a chat
history that handles this scenario naturally and makes the code more
readable.

Also now only previous research iterations that completed are
populated. Else they do not serve much purpose.
 2025-06-09 23:46:12 -07:00
Debanjum
bd928b9f3c Handle unset agent slug, name. E.g when chat with user created agents 2025-06-09 18:11:25 -07:00
Debanjum
5dd8a9cb24 Only add cache control to last Claude text block if exists, non-empty 
Otherwise Claude API throws error
 2025-06-08 19:41:21 -07:00
Debanjum
d638a49cd9 Release Khoj version 1.42.2 2025-06-07 13:32:12 -07:00
Debanjum
2423db0186 Remove broken link to deprecated summarize slash command in docs 2025-06-07 13:31:21 -07:00
Debanjum
b6ceaeeffc Execute doc search in parallel using asyncio instead of threadpool 2025-06-07 13:06:49 -07:00
Debanjum
dc1c3561fe Make search type comparison in document search more robust 2025-06-07 12:52:10 -07:00
Debanjum
b9c6252a4a Increase scroll amount on horizontal scroll in computer environment 2025-06-07 11:17:52 -07:00
Debanjum
3fc175d27b Restrict Khoj to work with python <3.13 
Python 3.13 not supported by all dependencies yet
 2025-06-07 00:44:04 -07:00
Debanjum
1bbf719b04 Apply migrations to db for test runs to install pgvector 2025-06-06 15:47:28 -07:00
Debanjum
77caf183ee Patch update django, next.js dependencies 2025-06-06 15:39:39 -07:00
Debanjum
c4cc70bcc9 Delete file summarization slash commands docs page 
File summarization slash commands have been deprecated. Folks can
upload files and ask their questions directly
 2025-06-06 15:30:21 -07:00
Debanjum
257c238a88 Improve DB clean up after test runs 2025-06-06 15:09:39 -07:00
Debanjum
6ac1530816 More robustly default to searching all content type 2025-06-06 15:09:39 -07:00
Debanjum
b21706aa45 Drop help, summarize and automation /slash commands from chat api 
Clean non useful slash commands to make chat API more maintanable.
- App version, chat model via /help is visible in other parts of the
  UX. Asking help questions with site:docs.khoj.dev filter isn't used
  or known to folks
- /summarize is esoterically tuned. Should be rewritten if add back.
  It wasn't being used by /research already
- Automations can be configured via UX. It wasn't being shown in UX
  already
 2025-06-06 15:09:39 -07:00
Debanjum
7f6db526c3 Enforce json for non reasoning anthropic models even in deepthought 2025-06-06 13:28:18 -07:00
Debanjum
d2c7e5516f Fix online chat actor tests, improve offline chat actor tests 
The chat actor (and director) tests haven't been looked into in a long
while. They'd gone stale in how they were calling thee functions. And
what was required to run them. Now the online chat actor tests work
again.
 2025-06-06 13:28:18 -07:00
Debanjum
2f4160e24b Use single extract questions method across all LLMs for doc search 
Using model specific extract questions was an artifact from older
times, with less guidable models.

New changes collate and reuse logic
- Rely on send_message_to_model_wrapper for model specific formatting.
- Use same prompt, context for all LLMs as can handle prompt variation.
- Use response schema enforcer to ensure response consistency across models.

Extract questions (because of its age) was the only tool directly within
each provider code. Put it into helpers to have all the (mini) tools
in one place.
 2025-06-06 13:28:18 -07:00
Debanjum
c2cd92a454 [Breaking] Move automation api into new router with consistent routes 
- Rename GET /api/automations to GET /api/automation
- Rename POST /api/trigger/automation to POST /api/automation/trigger
- Update calls to the automations API from the web app.
 2025-06-06 13:28:18 -07:00
Debanjum
7dfa710cb4 Log invalid automation ids for investigation and clean-up 2025-06-06 13:28:18 -07:00
Debanjum
7d59688729 Move document search tool into helpers module with other tools 
Document search (because of its age) was the only tool directly within
an api router. Put it into helpers to have all the (mini) tools in one
place.
 2025-06-06 13:28:18 -07:00
Debanjum
1dbe60a8a2 Give more readable name to document search tool 2025-06-06 13:28:18 -07:00
Debanjum
38fa34a861 Simplify ai provider converse methods 
- Add context based on information provided rather than conversation
  commands. Let caller handle passing appropriate context to ai
  provider converse methods
 2025-06-06 13:28:18 -07:00
Debanjum
bfd4695705 Save conversation in common chat api func instead of each ai provider 2025-06-06 13:28:18 -07:00
Debanjum
e7584bc29d Remove old "Notes" stop keyword for openai api based models 
They were when passing notes context to dumber models. Not required
for most models now.
 2025-06-06 13:28:18 -07:00
Debanjum
a9b1a26089 Update gunicorn default timeouts, workers. Configure via env vars 
Increase timeout to 180 (from 120s previous) and graceful timeout to
90 (from 30s default) to reduce

Increase default gunicorn workers and make it configurable to better
utilize (v)CPUs. This is manually configured (instead of using
multiprocessing.cpu_count()) as VMs/containers may read cpu count of
host machine instead of their VMs/containers.
 2025-06-06 13:28:18 -07:00
Debanjum
d16f9f272b Add ability to retry a query from the web app 2025-06-06 13:28:18 -07:00
Debanjum
05d4e19cb8 Pass deep typed chat history for more ergonomic, readable, safe code 
The chat dictionary is an artifact from earlier non-db chat history
storage. We've been ensuring new chat messages have valid type before
being written to DB for more than 6 months now.

Move to using the deeply typed chat history helps avoids null refs,
makes code more readable and easier to reason about.

Next Steps:
The current update entangles chat_history written to DB
with any virtual chat history message generated for intermediate
steps. The chat message type written to DB should be decoupled from
type that can be passed to AI model APIs (maybe?).

For now we've made the ChatMessage.message type looser to allow
for list[dict] type (apart from string). But later maybe a good idea
to decouple the chat_history recieved by send_message_to_model from
the chat_history saved to DB (which can then have its stricter type check)
 2025-06-04 00:03:14 -07:00
Debanjum
430459a338 Release Khoj version 1.42.1 2025-06-03 21:46:16 -07:00
Debanjum
f6e2eebecc Ignore devcontainer, launch.json from json pre-commit validation 
They follow jsonc format and allow comments but fail the json
validator. This is a spurious error and should be ignored
 2025-06-03 21:45:11 -07:00
Debanjum
d618f2d650 Raise value error if research pick next tool isn't a dictionary 
This will give a better error message with response content than the
failed to get errors from non dictionary response we were getting
earlier.
 2025-06-03 21:45:11 -07:00
Debanjum
65d9ad6cb2 Use tool calls to enforce response schema for anthropic models 
- Converts response schema into a anthropic tool call definition.
- Works with simple enums without needing to rely on $defs, $refs as
  unsupported by Anthropic API
- Do not force specific tool use as not supported with deep thought

This puts anthropic models on parity with openai, gemini models for
response schema following. Reduces need for complex json response
parsing on khoj end.
 2025-06-03 21:05:29 -07:00
Debanjum
d45d9d4cfb Fix malformed user uuids to fix automations [automations data loss] 
- **Malformed automations will be dropped**
  They can't run with malformed user uuid anyway.
 2025-06-03 21:05:29 -07:00
Debanjum
4892e73323 Remove unsuppported NUL char from file, chat before save to DB 2025-06-03 21:05:29 -07:00
Debanjum
27534f6533 Make query field in context optional 
Query field isn't set for all context. The current change was
preventing save to conversation errors when query unset in context.
 2025-06-03 19:59:05 -07:00
Debanjum
6e48f4de84 Fix to switch text to speech model via API 2025-06-03 19:59:05 -07:00
Debanjum
63a1a8e91f Try pre-install deps, use custom launch.json for dev container 
Previous attempts have not been sufficient. Let's see if this works
 2025-06-03 05:00:05 -07:00
Debanjum
29b973e748 Fix building server and web app dev container 2025-06-03 03:46:08 -07:00
Debanjum
50f37d541a Pre-install server deps for fast devcontainer start. Fix dev launch.json 
There seems to be a more standard mechanism of specifying launch.json
params for devcontainers. Previous mechanism to write launch.json to
.vscode/launch.json in post creation step does not work.

Improve default launch.json to include khoj admin username, password
with placeholder values to get started with local development faster.

Define dockerfile for devcontainer to pre-built server, web app
dependencies during dev container image creation stage. So install on
dev container startup is sped up as no need to install dependencies.
 2025-06-03 01:43:23 -07:00
Debanjum
f3a5fe1ae8 Release Khoj version 1.42.0 2025-06-01 20:52:25 -07:00
Debanjum
82ee0f5451 Revert computer dockerfile startup command to fix operating it 2025-06-01 20:39:58 -07:00
Debanjum
a236288ca9 Fixes to enable dockerized khoj to operate its computer 2025-06-01 19:19:01 -07:00
Debanjum
f95d352eb9 Ensure profile is right border aligned on khoj obsidian settings page 
On wide screens it wasn't taking up the header wasn't taking up the
full width, so profile picture could hang out in the middle somewhere.
 2025-06-01 17:02:08 -07:00
Debanjum
759ffc46b0 Default to read currently open file when chat with Khoj from Obsidian 
Vault is already indexed, this should ease engaging with current
context more easily.
 2025-06-01 16:56:19 -07:00
Debanjum
3fb8f77cd5 Fix terminal tool passed to claude 3.7 sonnet as anthropic operator 2025-06-01 16:55:17 -07:00
Debanjum
ddf028f7af Fix khoj computer image name used in docker-compose.yml instead 2025-06-01 16:44:28 -07:00
Debanjum
257bdfadef Setup vscode launch.json and configure pytests for dev container 2025-06-01 16:36:46 -07:00
Debanjum
a98525be01 Add default vscode config for khoj to ease development setup 2025-06-01 16:36:46 -07:00
Debanjum
c6cc709f62 Fix khoj computer image name and only build it once for each arch 2025-06-01 16:36:46 -07:00
Debanjum
a4eb85ac41 Reduce (superficial) xdg dir permissions errors on khoj computer start 2025-06-01 16:36:20 -07:00
sabaimran
e9a107cc06 fix spelling of development 2025-06-01 13:41:39 -07:00
Henri Jamet
dbfac89a0c Major updates to Obsidian Khoj plugin chat interface and editing features (#1109) 
## Description
This PR introduces significant improvements to the Obsidian Khoj
plugin's chat interface and editing capabilities, enhancing the overall
user experience and content management functionality.

## Features

### 🔍 Enhanced Communication Mode
I've implemented radio buttons below the chat window for easier
communication mode selection. The modes are now displayed as emojis in
the conversation for a cleaner interface, replacing the previous
text-based system (e.g., /default, /research). I've also documented the
search mode functionality in the help command.

#### Screenshots
- Radio buttons for mode selection
- Emoji display in conversations
![Recording 2025-02-11 at 18 56
10](https://github.com/user-attachments/assets/798d15df-ad32-45bd-b03f-581f6093575a)

### 💬 Revamped Message Interaction
I've redesigned the message buttons with improved spacing and color
coding for better visual differentiation. The new edit button allows
quick message modifications - clicking it removes the conversation up to
that point and copies the message to the input field for easy editing or
retrying questions.

#### Screenshots
- New message styling and color scheme
![Recording 2025-02-11 at 18 44
48](https://github.com/user-attachments/assets/159ece3d-2d80-4583-a7a8-2ef1f253adcc)
- Edit button functionality
![Recording 2025-02-11 at 18 47
52](https://github.com/user-attachments/assets/82ee7221-bc49-4088-9a98-744ef74d1e58)

### 🤖 Advanced Agent Selection System
I've added a new chat creation button with agent selection capability.
Users can now choose from their available agents when starting a new
chat. While agents can't be switched mid-conversation to maintain
context, users can easily start fresh conversations with different
agents.

#### Screenshots
- Agent selection dropdown
![Recording 2025-02-11 at 18 51
27](https://github.com/user-attachments/assets/be4208df-224c-45bf-a5b4-cf0a8068b102)

### 👁️ Real-Time Context Awareness
I've added a button that gives Khoj access to read Obsidian opened tabs.
This allows Khoj to read open notes and track changes in real-time,
maintaining a history of previous versions to provide more contextual
assistance.

#### Screenshots
- Window access toggle
![Recording 2025-02-11 at 18 59
01](https://github.com/user-attachments/assets/b596bfca-f622-41b7-b826-25a8e254d4a2)

### ✏️ Smart Document Editing
Inspired by Cursor IDE's intelligent editing and ChatGPT's Canvas
functionality, I've implemented a first version of a content creation
system we've been discussing. Using a JSON-based modification system,
Khoj can now make precise changes to specific parts of files, with
changes previewed in yellow highlighting before application.
Modification code blocks are neatly organized in collapsible sections
with clear action summaries. While this is just a first step, it's
working remarkably well and I have several ideas for expanding this
functionality to make Khoj an even more powerful content creation
assistant.

#### Screenshots
- JSON modification preview
- Change highlighting system
- Collapsible code blocks
- Accept/cancel controls
![Recording 2025-02-11 at 19 02
32](https://github.com/user-attachments/assets/88826c9e-d0c9-40da-ab78-9976c786aa9e)

---------

Co-authored-by: Debanjum <debanjum@gmail.com>
 2025-06-01 10:42:36 +05:30
Debanjum
dee767042e Operate Computer with Khoj Operator (#1190) 
## Summary
- Enable Khoj to operate computers: Add experimental computer operator
functionality that allows Khoj to interact with desktop environments,
browsers, and terminals to accomplish complex tasks
- Multi-environment support: Implement computer environments with GUI,
file system, and terminal access. Can control host computer or Docker
container computer

## Key Features
### Computer Operation Capabilities
- Desktop control (screenshots, clicking, typing, keyboard shortcuts)
- File editing and management
- Terminal/bash command execution
- Web browser automation
- Visual feedback via train-of-thought video playback

### Infrastructure & Architecture:
- Docker container (ghcr.io/khoj-ai/computer:latest) with Ubuntu 24.04,
XFCE desktop, VNC access
- Local computer environment support with pyautogui
- Modular operator agent system supporting multiple environment types
- Trajectory compression and context management for long-running tasks

###  Model Integration:
- Anthropic models only (Claude Sonnet 4, Claude 3.7 Sonnet, Claude Opus
4)
- OpenAI and binary operator agents temporarily disabled
- Enhanced caching and context management for operator conversations

### User Experience:
- `/operator` command or just ask Khoj to use operator tool to invoke
computer operation
- Integrate with research mode for extended 30+ minute task execution
- Video of computer operation in train of thought for transparency

### Configuration
- Set `KHOJ_OPERATOR_ENABLED=True` in `docker-compose.yml`
- Requires Anthropic API key
- Computer container runs on port 5900 (VNC)
 2025-05-31 22:04:12 -07:00
Debanjum
fa2e370ce6 Document how to enable and use computer operator in operator readme 2025-05-31 21:41:23 -07:00
Debanjum
ceb1d82bf6 Create khoj computer via cloud build. Add computer to docker-compose.yml 2025-05-31 21:39:38 -07:00
Debanjum
68f7aae71c Install claude 4 sonnet, latest gemini 2.5s when configure on first run 2025-05-31 20:52:27 -07:00
Debanjum
b90b724f9a Disable openai, binary operator agents until they become useful 2025-05-31 20:51:08 -07:00
Debanjum
830a1af69e Render operator train of thought as video on web app to ease viewing 
- You can seek through the train of thought video of computer operation or
  follow it in live mode.
- Interleaves video with normal text thoughts.
- Video available of old interactions and currently streaming message.
 2025-05-31 20:51:08 -07:00
Debanjum
6821bd38ed Fix mypy typing errors in operator environment files 
- Add type guards for action.path in drag vs text editor actions
- Added type guards for Union type attribute access
- Fixed variable naming conflicts between drag and text editor cases
- Resolved remaining typing issues in OpenAI, Anthropic agents
- Type guard without requiring another code indent level
 2025-05-31 20:51:08 -07:00
Debanjum
c5c06a086e Fix, improve openai operator agent for interrupts, computer environment 
- Create reusable method to call model
- Fix to summarize messages on operator run.
- Mark assistant tool calls with role = assistant, not environment

- Try fix message format when load after interrupts.
  Does not work well yet
 2025-05-31 20:51:08 -07:00
Debanjum
f517566560 Improve invoking keybindings on computer always using lowercase keys 
Previously CTRL+A would get triggered instead of ctrl+a. CTRL+A is
equivalent to ctrl+shift+a. This isn't intended and should be
called directly when required.

Now key combos like ctrl+a on computer firefox etc. work as expected
 2025-05-31 20:51:08 -07:00
Debanjum
2558ac7f18 Show thinking and engage deep thought for gemini 2.5 model series 
Gemini models now show (a summary of) their thoughts. Stream this in
research mode, similar to how it is done already for claude, deepseek,
qwen etc.
 2025-05-31 20:51:08 -07:00
Debanjum
cecbfe35e2 Rename compile response into a private operator agents function 2025-05-31 20:51:08 -07:00
Debanjum
ded1db642c Get max context for user, operator model pair for context compression 2025-05-31 20:51:08 -07:00
Debanjum
7eaf0e80c5 Get max prompt size for given user, model via reusable functions 2025-05-31 20:51:08 -07:00
Debanjum
3797f03625 Log ai model usage on every call to get_chat_usage_metrics in debug mode 2025-05-31 20:51:08 -07:00
Debanjum
4cb900658d Cache system prompt, tools of anthropic operator agent for efficiency 2025-05-31 20:51:08 -07:00
Debanjum
928e5ee8ad Cache messages to anthropic models from chat actors for efficiency 2025-05-31 20:51:08 -07:00
Debanjum
0d1e6b0d53 Do not overwrite system_prompt for idempotent AI API calls retry 
Previously on tenacity retry the system_prompt could get overwritten
 2025-05-31 20:51:08 -07:00
Debanjum
e0ea151f20 Implement file editor and terminal tools, in-built in claude 
This should improve viewing, editing files and viewing terminal
command outputs by anthropic operator
 2025-05-31 20:51:08 -07:00
Debanjum
21bf7f1d6d Continue interrupted operator run with new query and previous context 
Track research and operator results at each nested iteration step
using python object references + async events bubbled up from nested
iterators.

Instantiates operator with interrupted operator messages from research
or normal mode.

Reflects actual interaction trajectory as closely as possible to agent
including conversation history, partial operator trajectory and new
query for fine grained, corrigible steerability.

Research mode continues with operator tool directly if previous
iteration was an interrupted operator run.
 2025-05-31 20:51:08 -07:00
Debanjum
de35d91e1d Pass previous trajectory to operator agents for context 2025-05-31 20:51:08 -07:00
Debanjum
864e0ac8b5 Simplify research iteration and main research function names 2025-05-31 20:51:08 -07:00
Debanjum
6c9d569a22 Fix to get user questions in chat history from user not khoj message 
Since partial state reload after interrupt drops Khoj messages. The
assumption that there will always be a Khoj message after a user
message is broken. That is, there can now be multiple user messages
preceding a Khoj user message now.

This change allow for user queries to still be extracted for chat
history even if no khoj message follow.
 2025-05-31 20:51:08 -07:00
Debanjum
b6aa77a6f5 Lookback 3 previous turns to select next tool, for questions history 2025-05-31 20:50:03 -07:00
Debanjum
d511cbfa34 Extract constructing question history into shared function for reuse 
Minor logic update to only include non image inferred queries for
gemini, anthropic models as well instead of just for openai models.

Apart from that the extracted function should be functionally same.
 2025-05-31 16:50:26 -07:00
Debanjum
da663e184c Type operator results. Enable storing, loading operator trajectories. 
We were passing operator results as a simple dictionary. Strongly
typing it makes sense as operator results becomes more complex.

Storing operator results with trajectory on interrupts will allow
restarting interrupted operator run with agent messages of interrupted
trajectory loaded into operator agents
 2025-05-31 16:50:26 -07:00
Debanjum
675fc0ad05 Decouple trajectory compression from `act'. Reuse func to call llm api 2025-05-31 16:50:26 -07:00
Debanjum
b027024c42 Handle failed operator agent calls to anthropic api more gracefully 
Add anthropic operator api call errors to trajectory instead of
erroring out of current operator run
 2025-05-31 16:50:26 -07:00
Debanjum
d54bfc19e5 Add trajectory compression to anthropic operator agent 
- Add compression parameters to base operator agent for reuse
- Increase default operator iterations
 2025-05-31 16:50:26 -07:00
Debanjum
cb451fa67c Put default summarize prompt into operator agent 
This allows:
- Each operator agent to own its summarization prompt. That it can
  tune if it wants
- The outer operator loop to pass an override summarize prompt when it
  invokes the summarize func but it does not have to
 2025-05-31 16:50:26 -07:00
Debanjum
99fdd91a01 Latch to bottom instantly and well when auto scroll chat stream on web 2025-05-31 16:50:26 -07:00
Debanjum
253656b634 Fix engaging anthropic api cache for operator trajectories. 
It had become broken at some point due to refactoring. The cache
control was getting added and removed right after in add_action_results

What we actually wanted to do is clear the old cache breakpoint and
put a new one at the latest operator tool result message.

This should improve operator speed and lower costs with anthropic
models.
 2025-05-31 16:50:26 -07:00
Debanjum
faecbdb7d8 Enable operators to use computers 2025-05-31 16:50:25 -07:00
Debanjum
771909f76a Implement docker computer environment for operator 
- Generalize building pyautogui into executable python code snippet.
  This should work across docker and local. And should be easier to
  extend to operate a remote computer over the network as well.

- Create dockerfile for pyautogui operate-able containerized computer
 2025-05-28 17:40:32 -07:00
Debanjum
e117f57f64 Implement local computer environment for operator 2025-05-28 17:40:32 -07:00
Debanjum
7eab87bfdf Generalize operator to operate multiple types of environment 
Previously it could only operate a (playwright) browser. Now
- The operator logic and naming has been updated assuming
  multiple environment types can be operated
- The operator entrypoint is now at __init__.py to simplify imports
  and the entrypoint function is called operate_environment
- All operator agents have been updated to select their system prompts
  and tools based on the environment they'll operate
 2025-05-27 19:01:36 -07:00
Debanjum
c0689b2740 Easily interrupt and redirect khoj's research direction via chat 
- Khoj can now save and restore research from partial state
  This triggers an interrupt that saves the partial research, then
  when a new query is sent it loads the previous partial research as
  context and continues utilizing with the new user query to orient
  its future research
- Support natural interrupt and send query behavior from web app
  This triggers an abort and send when a user sends a chat message
  while khoj is in the middle of some previous research.

This interrupt mechanism enables a more natural, interactive
research flow
 2025-05-27 17:57:21 -07:00
Debanjum
c9e6b8e88d Align expected types to actual returned types by AI APIs, operator 2025-05-26 00:39:06 -07:00
Debanjum
c1c1fc6265 Make send message validation more robust on web app 2025-05-26 00:35:10 -07:00
Debanjum
6cb512d9cf Support natural interrupt and send query behavior from web app 
- Just send your new query. If a query was running previously it'd
be interrupted and new query would start processing. This improves on
the previous 2 click interrupt and send ux.

- Utilizes partial research for interrupted query, so you can now
redirect khoj's research direction. This is useful if you need to
share more details, change khoj's research direction in anyway or
complete research. Khoj's train of thought can be helpful for this.
 2025-05-26 00:35:10 -07:00
Debanjum
2b7dd7401b Continue interrupt queries only after previous query written to DB 2025-05-26 00:35:10 -07:00
Debanjum
3cd6e1a9a6 Save and restore research from partial state 2025-05-26 00:35:09 -07:00
Debanjum
a83c36fa05 Validate operator, research, context.query fields of ChatMessage 
- Track operator, research context in ChatMessage
- Track query field in (document) context field of ChatMessage

This allows validating chat message before inserting into DB
 2025-05-26 00:03:59 -07:00
Debanjum
02ee4e90a2 Pass doc/web/code/operator context as list[dict] of message content 2025-05-26 00:03:59 -07:00
Debanjum
98b56316e4 Support constructing chat message as a list of dictionaries 
Research mode recently started passing iteration as list of message
content dicts. This change extends to storing it as is in DB.
 2025-05-26 00:03:59 -07:00
Debanjum
df9ab51fd0 Track research results as iteration list instead of iteration summaries 2025-05-26 00:03:59 -07:00
Debanjum
5d65fa8698 Use Django timezone funcs to make datetimes in DB timezone aware 
These seem to be a new class of errors showing up. Explicitly using
django timezone functions to add awareness to date time files stored
in DB seems to mitigate the issue.

Related #1180
 2025-05-25 23:43:06 -07:00
Debanjum
231aa1c0df Support claude 4 models. Engage reasoning, operator. Track costs etc. 
- Engage reasoning when using claude 4 models
- Allow claude 4 models as monolithic operator agents
- Ease identifying which anthropic models can reason, operate GUIs
- Track costs, set default context window of claude 4 models
- Handle stop reason on calls to new claude 4 models
 2025-05-25 23:43:06 -07:00
Debanjum
dca17591f3 Handle parsing json from string with plain text suffix 2025-05-23 19:44:02 -07:00
Debanjum
acebb90643 Mention keys expected in prompt to next research tool selector 2025-05-23 19:44:02 -07:00
Debanjum
e968cca273 Clean usage of conversation_id in chat API function 
- Normalize conversation_id type to str instead of str or UUID
- Do not pass conversation_id to agenerate_chat_response as
  the associated conversation is also being passed. So can get its id
  directly.
 2025-05-23 19:44:02 -07:00
Debanjum
a76032522e Add type hints to function args calling anthropic model api 2025-05-22 15:02:45 -07:00
Debanjum
97c5222b04 Set type hints and reorder args of all converse_[provider] methods 
- Query is more important and should be passed before references
- Add type hints to user query and references for code readability
 2025-05-22 15:02:45 -07:00
Debanjum
2ea16298aa Create Operator Framework. Enable Khoj to Operate Web Browser (#1174) 
## Overview

1. Create base framework to compose different operators and environments
for Khoj to operate.
2. Enable Khoj to operate a web browser using anthropic, openai, gemini
or open-source models

**Note**: *This is an alpha level feature release. It is meant for local
testing by contributors and self-hosters.*

## Capabilities
- Have Khoj operate a web browser to complete tasks that require actions
and visual feedback.
- Experiment with any vision model as operator. Khoj supports monolithic
and binary operator
- Monolithic operators rely on a single models like claude, openai to
both reason and ground operator actions
- Binary operators allow bootstrapping a fully local operator. It can
use any vision model for visual reasoning when paired with a capable
visual grounding model.

## Limitations
- In general, it is slower, more expensive and less comprehensive than
standard Khoj for research

## Setup
1. Install Khoj with playwright by either 
   - running `pip install khoj[local]`
- installing playwright separately via `pip install playwright` and
`playwright install chromium`
2. Set `KHOJ_OPERATOR_ENABLED` env var to true (i.e
`KHOJ_OPERATOR_ENABLED=true`)
3. Start Khoj (e.g `USE_EMBEDDED_DB="true" khoj --anonymous-mode -vv`)
4. Add the necessary chat model(s) with `vision enabled` via your [Khoj
Admin Panel](http://localhost:42110/server/admin)
- To use Anthropic claude: `claude-3.7-sonnet*` chat model is required
with vision enabled
- To use Openai operator: `gpt-4o` chat model is required with vision
enabled
- For other operator configurations: a chat model named `ui-tars-1.5` is
required with vision enabled
This can technically be any visual grounding model served via an openai
compatible api. I've just tested with ui-tars-1.5-7b deployed to an HF
inference endpoint for now. See [deployment
instructions](https://github.com/bytedance/UI-TARS/blob/main/README_deploy.md)
5. Set your desired vision chat model via [user
settings](http://localhost:42110/settings) to use as operator.
6. Run your queries with either the `/operator` slash command or by just
asking Khoj in your query to use the operator tool. You can combine run
operator in research mode a well

### Advanced Usage
- Reuse Browser Session
- Why: Have Khoj operate web services you've logged into. E.g manage
your gmail, github, social media etc.
  - Setup
1. Start Chromium or Edge in Remote Debugging mode. For example, on Mac
you can start Edge by running the following in your terminal:
`/Applications/Microsoft\ Edge.app/Contents/MacOS/Microsoft\ Edge
--remote-debugging-port=9222`
4. Connect Khoj to that browser instance by setting the environment
variable `KHOJ_CDP_URL` to its URL.
      By default you'd set `KHOJ_CDP_URL="http://localhost:9222"`

## Architecture
### Operator Agents
| Type | Design |
|----- |-----|
| Monolithic | <img
src="https://github.com/user-attachments/assets/7a96440f-1732-482b-9bd9-0920cb0c60890"
width=400> |
| Binary | <img
src="https://github.com/user-attachments/assets/c5d101c0-3475-43c2-a301-daa943cde190"
width=400> |
 2025-05-20 01:30:36 -07:00
Debanjum
19b4c18b69 Configure max iterations per operator run via environment variable 2025-05-20 01:03:11 -07:00
Debanjum
06a1a22e3b Align generic grounding agent's interface with uitars grounding agent 
The generic grounding agent has not been tested properly but at least
it should be aligned with the interface being used by the ui-tars
grounding agent which has been tested.
 2025-05-20 00:31:56 -07:00
Debanjum
0ce74e0329 Show operator context when use operator in default and research mode 2025-05-20 00:31:56 -07:00
Debanjum
cc355f93fc Use operator context consistently as a dict[str, str] of query, result 2025-05-20 00:31:56 -07:00
Debanjum
07e33994f0 Reduce scroll amount to have previous page stay a bit on screen 2025-05-20 00:31:56 -07:00
Debanjum
e2c1b1fcd3 Add dev container config to ease setup for remote development 2025-05-19 23:34:31 -07:00
Debanjum
fdb681ca0e Only install desktop, obsidian app from dev_setup.sh with --full flag 2025-05-19 23:34:31 -07:00
Debanjum
33dd4c8c33 Handle gemini returning simple string in response candidates 2025-05-19 19:45:10 -07:00
Debanjum
626ced8b8b Fix adding code results to chatml messages context 2025-05-19 19:45:10 -07:00
Debanjum
ded753ff9a Improve parsing tool use coordinate returned by claude operator agent 
It sometimes outputs coordinates in string rather than list. Make
parser more robust to those kind of errors.

Share error with operator agent to fix/iterate on instead of exiting
the operator loop.
 2025-05-19 16:28:55 -07:00
Debanjum
473dd006d5 Remove unnecessary images conversion to png in binary operator agent. 
It's handled by the ai model interaction handlers in khoj server core.
 2025-05-19 16:28:55 -07:00
Debanjum
9f3fbf9021 Encourage reasoner, grounder to work better together in binary operator 
- Encourage grounder to adhere to the reasoners action instruction
- Encourage reasoner to explore other actions when stuck in a loop
  Previously seemed to be forcing it too strongly to choose
  "single most important" next action. So may not have been exploring
  other actions to achieve objective on initial failure.
 2025-05-19 16:28:55 -07:00
Debanjum
ac19f6d336 Improve operator exception handling 
- Do not catch errors messages just to re-throw them. Results in
  confusing exception happened during handling of an exception
  stacktrace. Makes it harder to debug

- Log error when action_results.content isn't set or empty to
  debug this operator run error
 2025-05-19 16:28:55 -07:00
Debanjum
59e0e092b0 Remove deprecated prompt for grounding model to choose goto, back func 
Goto and back functions are chosen by the visual reasoning model for
increased reliability in selecting those tools. The ui-tars grounding
models seems too tuned to use a specific set of tools.
 2025-05-19 16:28:55 -07:00
Debanjum
1442a4f6fb Handle reasoning messages returned by openai cua model 
Documentation about this is currently limited, confusing. But it seems
like reasoning item should be kept if computer_call after, else drop.

Add noop placeholder for reasoning item to prevent termination of
operator run on response with just reasoning.
 2025-05-19 16:28:55 -07:00
Debanjum
95f211d03c Resolve mypy typing errors in operator code 2025-05-19 16:28:55 -07:00
Debanjum
33689feb91 Handle more openai response types for better rendering and error avoidance 
The reasoning messages in openai cua needs to be passed back or some
such. Else it throws missing response with required id error.

Folks are confused about expected behavior for this online as well.
The documentation to handle this seems to be sparse, unclear.
 2025-05-19 16:28:55 -07:00
Debanjum
3a75cd3c3d Only trigger claude, openai monolithic operators with specific models 
To use Anthropic monolithic operator, set chat model to claude-3.7-sonnet
To use Openai monolithic operator, set chat model to gpt-4o
 2025-05-19 16:28:55 -07:00
Debanjum
258b5a0372 Show operator screenshots with reasoning in train of thought on web app 2025-05-19 16:28:55 -07:00
Debanjum
21a9556b06 Show formatted action, env screenshot after action on each operator step 
Show natural language, formatted text for each action. Previously we
were just showing json dumps of the actions taken.

Pass screenshot at each step for openai, anthropic and binary operator agents
Use text and image field in json passed to client for rendering both.

Show actions, env screenshot after actions applied in train of thought.
Showing the post action application screenshot seems more intuitive.

Previously we were showing the screenshot used to decide next action.
This pre action application screenshot was being shown after next
action decided (in train of thought). This was anyway misleading to
the actual ordering of event.

Rendered response is now a structured payload (dict) passing image
and text to be rendered up from operator to clients for rendering of
train of thought.
 2025-05-19 16:28:55 -07:00
Debanjum
a1d712e031 Add current cursor position to browser screenshots for ai, human view 2025-05-19 16:28:55 -07:00
Debanjum
1be3986537 Require explicit switch to enable operator locally for now 
Operator is still early in development. To enable it:
- Set KHOJ_OPERATOR_ENABLE environment variable to true
- Run any one of the commands below:
  - `pip install khoj[local]'
  - `pip install khoj[dev]'
  - `pip install playwright'
 2025-05-19 16:28:55 -07:00
Debanjum
b395a438d0 Fix handling multiple actions requested by grounding agent in an iteration 2025-05-19 16:28:55 -07:00
Debanjum
e5415bdaee Only reasoning agent should terminate run, not the grounding agent. 
Grounding agent does not have the full context and capabilities to
make this call. Only let reasoning agent make termination decision.

Add a wait action instead when grounder requests termination.
 2025-05-19 16:28:55 -07:00
Debanjum
ffe58d2ec1 Parse goto, back actions directly from instruction for uitars grounder 
UI tars grounder doesn't like calling non-standard functions like
goto, back.

Directly parse visual reasoner instruction to bypass uitars grounder
model.

At least for goto and back functions grounding isn't necessary, so
this works well.
 2025-05-19 16:28:55 -07:00
Debanjum
7395af3c3a Allow visual grounder of binary operator agent to see past actions 
Previously the grounding agent would be reset on every call. So it
only saw the most recent instruction and screenshot to make its next
action suggestion.

This change allows the visual grounders to see past instructions and
actions to prevent looping and encourage more exploratory action
suggestions by it when stuck or see errors.
 2025-05-19 16:28:55 -07:00
Debanjum
d8bc6239f8 Bifurcate visual grounder into a ui-tars specific & generic grounder 
Split visual grounder into two implementations:

- A ui-tars specific visual grounder agent. This uses the canonical
  implementation of ui-tars with specialized system prompt and action
  parsing.

- Fallback to generic visual grounder utilizing tool-use and served over
  any openai compatible api. This was previously being used for our
  ui-tars implementation as well.
 2025-05-19 16:28:55 -07:00
Debanjum
c3bfb15fab Support KeyUp, KeyDown operator actions. Make coordinates into floats 2025-05-19 16:28:55 -07:00
Debanjum
b279060e2c Enable using Operator with Gemini models 2025-05-19 16:28:55 -07:00
Debanjum
0d8fb667ec Add action results for multiple actions similar to other operator agents 
Adds the results of each action in a separate item in message content.
Previously we were adding this as a single larger text blob. This
changes adds structure to simplify post processing (e.g truncation).

The updated add_action_results should also require less work to
generalize if we pass tool call history to grounding model as
action results in valid openai format.
 2025-05-19 16:28:55 -07:00
Debanjum
e17c06b798 Set operator query on init. Pass summarize prompt to summarize func 
The initial user query isn't updated during an operator run. So set it
when initializing the operator agent. Instead of passing it on every
call to act.

Pass summarize prompt directly to the summarize function. Let it
construct the summarize message to query vision model with.
Previously it was being passed to the add_action_results func as
previous implementation that did not use a separate summarize func.

Also rename chat_model to vision_model for a more pertinent var name.

These changes make the code cleaner and implementation more readable.
 2025-05-19 16:28:55 -07:00
Debanjum
38bcba2f4b Make back action in browser environment use goto to avoid timeouts 
For some reason the page.go_back() action in playwright had a much
higher propensity to timeout. Use goto instead to reduce these page
traversal timeouts.

This requires tracking navigation history.
 2025-05-19 16:28:55 -07:00
Debanjum
fd139d4708 Improve termination on task completion for binary operator agent 
Only let the visual reasoner handle terminating the operator run.
Previously the grounder was also able to trigger termination.

Make catching the termination by the reasoner more robust
 2025-05-19 16:28:55 -07:00
Debanjum
680c226137 Use any supported vision model as reasoner for binary operator agent 2025-05-19 16:28:55 -07:00
Debanjum
3839d83b90 Modularize operator into separate files for agent, action, environment etc 
The previous browser_operator.py file had become pretty massive and
unwieldy. This change breaks it apart into separate files for
- the abstract environment and operator agent base
- the concrete agents: anthropic, openai and binary
- the concrete environment browser operator
- the operator actions used by agents and environment
 2025-05-19 16:28:55 -07:00
Debanjum
833c8ed150 Add a flexible operator agent using separate reasoning, grounder models 
- This operator works with model served over an openai compatible api
- It uses separate vision models to reason and ground actions.

This improves flexibility in the operator agents that can be created.
We do not know need our operator agent ot rely on monolithic models to
can both reason over visual data and ground their actions.

We can create operator agent from 2 separate models:
1. To reason over screenshots to suggest natural language next action
2. To ground those suggestion into visually grounded actions

This allows us to create fully local operators or operators combining
the best visual reasoner with the best visual grounder models.
 2025-05-19 16:28:55 -07:00
Debanjum
773d20a26f Improve instructions to the openai operator agent. 
Inform it can only control a single playwright browser page.
Previously it was assuming it is operating a whole browser, so would
have trouble navigating to different pages.

Improve handling of error in action parsing
 2025-05-19 16:28:55 -07:00
Debanjum
4db888cd62 Simplify operator loop. Make each OperatorAgent manage state internally. 
Remove each OperatorAgent specific code from leaking out into the
operator. The Oprator just calls the standard OperatorAgent functions.

Each AgentOperator specific logic is handled by the OperatorAgent
internally.

The improve the separation of responsibility between the operator,
OperatorAgent and the Environment.

- Make environment pass screenshot data in agent agnostic format
  - Have operator agents providers format image data to their AI model
    specific format
  - Add environment step type to distinguish image vs text content
- Clearly mark major steps in the operator iteration loop
- Handle anthropic models returning computer tool actions as normal
  tool calls by normalizing next action retrieval from response for it
- Remove unused ActionResults fields
- Remove unnnecessary placeholders to content of action results like
  for screenshot data
 2025-05-19 16:28:55 -07:00
Debanjum
a1c9c6b2e3 Add pages visited via browser operator to references returned to clients 2025-05-19 16:28:55 -07:00
Debanjum
e71575ad1a Render screenshot in train of thought on openai agent screenshot action 2025-05-19 16:28:55 -07:00
Debanjum
78e052bfcb Decouple environment from operator agent to improve modularity 
Decouple applying action on Environment from next action decision by
OperatorAgent

- Create an abstract Environment class with a `step' method
  and a standardized set of supported actions for each concrete Environment
  - Wrap playwright page into a concrete Environment class

- Create abstract OperatorAgent class with an abstract `act' method
  - Wrap Openai computer Operator into concrete OperatorAgent class
  - Wrap Claude computer Operator into a concrete OperatorAgent class

Handle interaction between Agent's action
 2025-05-19 16:28:55 -07:00
Debanjum
7c60e04efb Pull out common iteration loop into main browser operator method 2025-05-19 16:28:54 -07:00
Debanjum
08e93c64ab Render screenshot in train of thought on browser screenshot action 
Update web app to render screenshot image when screenshot action taken
by browser operator
 2025-05-19 16:28:54 -07:00
Debanjum
188b3c85ae Force open links in current page to stay in operator page context 
Previously some link clicks would open in new tab. This is out of the
browser operator's context and so the new page cannot be interacted
with by the browser operator.

This change catches new page opens and opens them in the context page
instead.
 2025-05-19 16:28:54 -07:00
Debanjum
20f87542e5 Add cancellation support to browser operator via asyncio.Event 2025-05-19 16:28:54 -07:00
Debanjum
9f75622346 Allow browser operator to use browser with existing context over CDP 
Give the Khoj browser operator access to browser with existing
context (auth, cookies etc.) by starting it with CDP enabled.

Process:
1. Start Browser with CDP enabled:
  `Edge/Chromium/Chrome --remote-debugging-port=9222'
2. Set the KHOJ_CDP_URL env var to the CDP url of the browser to use.
3. Start Khoj and ask it to get browser based work done with operator
   + research mode
 2025-05-19 16:28:54 -07:00
Debanjum
b9ea538b02 Support operating web browser with Anthropic models 
- Add back() and goto(url) helper functions to operate browser
- Cache operator messages to Anthropic API for speed and cost savings
 2025-05-19 16:28:54 -07:00
Debanjum
2e86141575 Enable Khoj to use a GUI web browser. Operate it with Openai models 2025-05-19 16:28:54 -07:00
Debanjum
ab5d0b5878 Upgrade server dependencies 2025-05-19 16:28:21 -07:00
Debanjum
22cd638add Fix handling unset openai_base_url to run eval with openai chat models 
The github run_eval workflow sets OPENAI_BASE_URL to empty string.

The ai model api created during initialization for openai models gets
set to empty string rather than None or the actual openai base url

This tries to call llm at to empty string base url instead of the
default openai api base url, which obviously fails.

Fix is to map empty base url's to the actual openai api base url.
 2025-05-19 16:19:43 -07:00
Debanjum
cf55582852 Retry on empty response or error in chat completion by llm over api 
Previously all exceptions were being caught. So retry logic wasn't
getting triggered.

Exception catching had been added to close llm thread when threads
instead of async was being used for final response generation.

This isn't required anymore since moving to async. And we can now
re-enable retry on failures.

Raise error if response is empty to retry llm completion.
 2025-05-19 11:27:19 -07:00
Debanjum
7827d317b4 Widen vision support for chat models served via openai compatible api 
Send image as png to non-openai models served via an openai compatible
api. As more models support png than webp.

Continue storing images as webp on server for efficiency.

Convert to png at the openai api layer and only for non-openai models
served via an openai compatible api.

Enable using vision models like ui-tars (via llama.cpp server), grok.
 2025-05-19 11:27:19 -07:00
Debanjum
4f3fdaf19d Increase khoj api response timeout on evals call. Handle no decision 2025-05-18 19:14:49 -07:00
Debanjum
31dcc44c20 Output tokens >> reasoning tokens to avoid early response termination. 2025-05-18 14:45:23 -07:00
Debanjum
73e28666b5 Fix to set default chat model for all user tiers via env var 2025-05-18 14:45:23 -07:00
Debanjum
06dcd4426d Improve Research Mode Context Management (#1179) 
### Major
* Do more granular truncation on hitting context limits
* Pack research iterations as list of message content instead of
separate messages
* Update message truncation logic to truncate items in message content
list
* Make researcher aware of number of web, doc queries allowed per
iteration

### Minor
* Prompt web page reader to extract quantitative data as is from pages
* Track gemini 2.0 flash lite cost. Reduce max prompt size for 4o-mini
* Ensure time to first token logged only once per chat response
* Upgrade tenacity to respect min_time passed to exponential backoff
with jitter function
 2025-05-17 17:38:31 -07:00
Debanjum
fd591c6e6c Upgrade tenacity to respect min time for exponential backoff 
Fix for issue is in tenacity 9.0.0. But older langchain required
tenacity <0.9.0.

Explicitly pin version of langchain sub packages to avoid indexing
and doc parsing breakage.
 2025-05-17 17:37:15 -07:00
Debanjum
988bde651c Make researcher aware of no. of web, doc queries allowed per iteration 
- Construct tool description dynamically based on configurable query
  count
- Inform the researcher how many webpage reads, online searches and
  document searches it can perform per iteration when it has to decide
  which next tool to use and the query to send to the tool AI.
- Pass the query counts to perform from the research AI down to the
  tool AIs
 2025-05-17 17:37:15 -07:00
Debanjum
417ab42206 Track gemini 2.0 flash lite cost. Reduce max prompt size for 4o-mini 2025-05-17 17:37:15 -07:00
Debanjum
e125e299a7 Ensure time to first token logged only once per chat response 
Time to first token Log lines were shown multiple times if new chunk
bein streamed was empty for some reason.

This change makes the logic robust to empty chunks being recieved.
 2025-05-17 17:37:15 -07:00
Debanjum
2694734d22 Update truncation logic to handle multi-part message content 2025-05-17 17:37:15 -07:00
Debanjum
a337d9e4b8 Structure research iteration msgs for more granular context management 
Previously research iterations and conversation logs were added to a
single user message. This prevented truncating each past iteration
separately on hitting context limits. So the whole past research
context had to be dropped on hitting context limits.

This change splits each research iteration into a separate item in a
message content list.

It uses the ability for message content to be a list, that is
supported by all major ai model apis like openai, anthropic and gemini.

The change in message format seen by pick next tool chat actor:
- New Format
  - System: System Message
  - User/Assistant: Chat History
  - User: Raw Query
  - Assistant: Iteration History
    - Iteration 1
    - Iteration 2
  - User: Query with Pick Next Tool Nudge

- Old Format
  - User: System + Chat History + Previous Iterations Message
  - User: Query

- Collateral Changes
The construct_structured_message function has been updated to always
return a list[dict[str, Any]].

Previously it'd only use list if attached_file_context or vision model
with images for wider compatibility with other openai compatible api
 2025-05-17 17:37:15 -07:00
Debanjum
0f53a67837 Prompt web page reader to extract quantitative data as is from pages 
Previously the research agent would have a hard time getting
quantitative data extracted by the web page reader tool AI.

This change aims to encourage the web page reader tool to extract
relevant data in verbatim form for higher granularity research and
responses.
 2025-05-17 17:37:15 -07:00
Debanjum
99a2305246 Improve tool chat history constructor and fix its usage during research. 
Code tool should see code context and webpage tool should see online
context during research runs

Fix to include code context from past conversations to answer queries.

Add all queries to tool chat history when no specific tool to limit
extracting inferred queries for provided.
 2025-05-17 17:37:15 -07:00
Debanjum
8050173ee1 Timeout calls to khoj api in evals to continue to next question 2025-05-17 17:37:11 -07:00
Debanjum
442c7b6153 Retry running code on more request exception 2025-05-17 17:37:11 -07:00
Debanjum
10a5d68a2c Improve retry, increase timeouts of gemini api calls 
- Catch specific retryable exceptions for retry
- Increase httpx timeout from default of 5s to 20s
 2025-05-17 16:38:55 -07:00
Debanjum
20f08ca564 Reduce timeouts on calling local and online llms via openai api 
- Use much larger read, connect timeout if llm served over local url
- Use larger timeout duration than default (5s) for online llms too
  This matches timeout duration increase calls to gemini api
 2025-05-17 16:38:55 -07:00
Debanjum
e0352cd8e1 Handle unset ttft in metadata of failed chat response. Fixes evals. 
This was causing evals to stop processing rest of batch as well.
 2025-05-17 15:06:22 -07:00
Debanjum
673a15b6eb Upgrade hf hub package to include hf_xet for faster downloads 2025-05-17 15:06:22 -07:00
Debanjum
d867dca310 Fix send_message_to_model_wrapper by using sync is_user_subscribed check 
Calling an async function from a sync function wouldn't work.
 2025-05-17 15:06:22 -07:00
Sajjad Baloch
a4ab498aec Update README for better contributions (#1170) 
- Improve overall flow of the contribute section of Readme
- Fix where to look for good first issues. The contributors board is outdated. Easier to maintain and view good-first-issue with issue tags directly.

Co-authored-by: Debanjum <debanjum@gmail.com>
 2025-05-12 09:51:01 -06:00
Debanjum
2feed544a6 Add Gemini 2.0 flash back to default gemini chat models list 
Remove once gemini 2.5 flash is GA
 2025-05-11 19:05:09 -06:00
Debanjum
2e290ea690 Pass conversation history to generate non-streaming chat model responses 
Allows send_message_to_model_wrapper func to also use conversation
logs as context to generate response. This is an optional parameter
 2025-05-09 00:02:14 -06:00
Debanjum
8787586e7e Dedupe code to format messages before sending to appropriate chat model 
Fallback to assume not a subscribed user if user not passed.
This allows user arg to be actually optional in the async
send_message_to_model_wrapper function
 2025-05-09 00:02:14 -06:00
Debanjum
e94bf00e1e Add cancellation support to research mode via asyncio.Event 2025-05-09 00:01:45 -06:00
Debanjum
1572781946 Parse and show reasoning model thoughts (#1172) 
### Major
All reasoning models return thoughts differently due to lack of
standardization.
We normalize thoughts by reasoning models and providers to ease handling
within Khoj.

The model thoughts are parsed during research mode when generating final
response.
These model thoughts are returned by the chat API and shown in train of
thought shown on web app.

Thoughts are enabled for Deepseek, Anthropic, Grok and Qwen3 reasoning
models served via API.
Gemini and Openai reasoning models do not show their thoughts via
standard APIs.

### Minor
- Fix ability to use Deepseek reasoner for intermediate stages of chat
- Enable handling Qwen3 reasoning models
 2025-05-02 20:29:38 -06:00
Debanjum
2cd7302966 Parse Grok reasoning model thoughts returned by API 2025-05-02 19:59:17 -06:00
Debanjum
8cadb0dbc0 Parse Anthropic reasoning model thoughts returned by API 2025-05-02 19:59:13 -06:00
Debanjum
ae4e352b42 Fix formatting to use Deepseek reasoner for completion via OpenAI API 
Previously Deepseek reasoner couldn't be used via API for completion
because of the additional formatting constrains it required was being
applied in this function.

The formatting fix was being applied in the chat completion endpoint.
 2025-05-02 19:11:16 -06:00
Debanjum
61a50efcc3 Parse DeepSeek reasoning model thoughts served via OpenAI compatible API 
DeepSeek reasoners returns reasoning in reasoning_content field.

Create an async stream processor to parse the reasoning out when using
the deepseek reasoner model.
 2025-05-02 19:11:16 -06:00
Debanjum
16f3c85dde Handle thinking by reasoning models. Show in train of thought on web client 2025-05-02 19:11:16 -06:00
Debanjum
d10dcc83d4 Only enable reasoning by qwen3 models in deepthought mode 2025-05-02 18:36:49 -06:00
Debanjum
6eaf54eb7a Parse Qwen3 reasoning model thoughts served via OpenAI compatible API 
The Qwen3 reasoning models return thoughts within <think></think> tags
before response.

This change parses the thoughts out from final response from the
response stream and returns as structured response with thoughts.

These thoughts aren't passed to client yet
 2025-05-02 18:36:45 -06:00
Debanjum
7b9f2c21c7 Parse thoughts from thinking models served via OpenAI compatible API 
OpenAI API doesn't support thoughts via chat completion by default.
But there are thinking models served via OpenAI compatible APIs like
deepseek and qwen3.

Add stream handlers and modified response types that can contain
thoughts as well apart from content returned by a model.

This can be used to instantiate stream handlers for different model
types like deepseek, qwen3 etc served over an OpenAI compatible API.
 2025-05-02 17:49:16 -06:00
Debanjum
6843db1647 Use conversation specific chat model to respond to free tier users 
Recent changes enabled free tier users to switch free tier chat models
per conversation or the default.

This change enables free tier users to generate responses with their
conversation specific chat model.

Related: #725, #1151
 2025-05-02 17:48:48 -06:00
Debanjum
5b5efe463d Remove inline base64 images from webpages read with Firecrawl 2025-05-02 14:11:27 -06:00
Debanjum
559b323475 Support attaching jupyter/ipython notebooks from the web app to chat 2025-05-02 14:11:27 -06:00
sabaimran
dab6977fed add number 1 repo of day badge 2025-04-23 16:49:12 -07:00
Debanjum
964a784acf Release Khoj version 1.41.0 2025-04-23 19:01:27 +05:30
Debanjum
23dae72420 Update default models: Gemini models to 2.5 series, Gpt 4o to 4.1 2025-04-23 18:40:38 +05:30
Debanjum
d84a0f6e2c Use latest node base image to build web app for khoj docker image 2025-04-23 17:53:33 +05:30
Debanjum
dd46bcabc2 Track gpt-4.1 model costs. Set prompt size of new gemini, openai models 2025-04-23 17:53:33 +05:30
Debanjum
87262d15bb Save conversation to DB in the background, as an asyncio task 2025-04-22 17:42:33 +05:30
Debanjum
f929ff8438 Simplify AI Chat Response Streaming (#1167) 
Reason
---
- Simplify code and logic to stream chat response by solely relying on
asyncio event loop.
- Reduce overhead of managing threads to increase efficiency and
throughput (where possible).

Details
---
- Use async/await with no threading when generating chat response via
OpenAI, Gemini, Anthropic AI model APIs
- Use threading for offline chat model as llama-cpp doesn't support
async streaming yet
 2025-04-21 14:28:02 +05:30
Debanjum
a4b5842ac3 Remove ThreadedGenerator class, previously used to stream chat response 2025-04-21 14:16:40 +05:30
Debanjum
763fa2fa79 Refactor Offline chat response to stream async, with separate thread 2025-04-21 10:48:38 +05:30
Debanjum
932a9615ef Refactor Anthropic chat response to stream async, no separate thread 2025-04-21 10:46:07 +05:30
Debanjum
a557031447 Refactor Gemini chat response to stream async, no separate thread 2025-04-21 10:46:07 +05:30
Debanjum
0751f2ea30 Refactor Openai chat response to stream async, no separate thread 
- Refactor chat API to use async/await for Openai streaming
- Fix and clean Openai chat response async streaming
 2025-04-21 10:44:49 +05:30
Debanjum
c93c0d982e Create async get anthropic, openai client funcs, move to reusable package 
This package is where the get openai client functions also reside.
 2025-04-21 09:30:26 +05:30
Debanjum
973aded6c5 Fix system prompt to make openai reasoning models md format response 2025-04-20 20:33:45 +05:30
Debanjum
21d19163ba Just pass user rather than whole request object to doc search func 2025-04-20 20:33:45 +05:30
Debanjum
b2390fa977 Allow attaching typescript files to chat on web app 2025-04-19 19:08:11 +05:30
Debanjum
4d331e5ad2 Bump documentation dependencies 2025-04-19 18:38:31 +05:30
Debanjum
d6aafef464 Fix formatting of FAQ section in README.md 2025-04-19 18:31:16 +05:30
Debanjum
8f9090940b Resolve datetime utcnow deprecation warnings (#1164) 
# PR Summary
This small PR resolves the deprecation warnings on `datetime` in
Python3.12+. You can find them in the [CI
logs](https://github.com/khoj-ai/khoj/actions/runs/14538833837/job/40792624987#step:9:134):
```python
  /__w/khoj/khoj/src/khoj/processor/content/images/image_to_entries.py:61: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    timestamp_now = datetime.utcnow().timestamp()
```
 2025-04-19 18:26:52 +05:30
Debanjum
5441793a10 Allow AI model switching based on User Tier (#1151) 
Overview
---
Enable free tier users to chat with any AI model made available on free tier 
of production deployments like [Khoj cloud](https://app.khoj.dev).

Previously model switching was completely disabled for users on free tier.

Details
---
- Track price tier of each Chat, Speech, Image, Voice AI model in DB
- Update API to allow free tier users to switch between free models
- Update web app to allow model switching on agent creation, settings
  chat page (via right side pane), even for free tier users.
 2025-04-19 18:14:37 +05:30
Debanjum
ab29ffd799 Fix web app packaging for pypi since upgrade to python 3.11.12 in CI 2025-04-19 18:03:29 +05:30
Debanjum
79fc911633 Enable free tier users to switch between free tier AI models 
- Update API to allow free tier users to switch between free models
- Update web app to allow model switching on agent creation, settings
  chat page (via right side pane), even for free tier users.

Previously the model switching APIs and UX fields on web app were
completely disabled for free tier users
 2025-04-19 17:29:53 +05:30
Debanjum
30570e3e06 Track Price tier for each Chat, Speech, Image, Voice AI model in DB 
Enables users on free plan to choose AI models marked for free tier
 2025-04-19 09:44:33 +05:30
Debanjum
fdaf51f0ea Fix formatting in readme and documentation 2025-04-19 00:36:47 +05:30
Emmanuel Ferdman
fee1d3682b Resolve datetime deprecation warnings 
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
 2025-04-18 10:41:16 -07:00
Debanjum
eb1406bcb4 Support deepthought in research mode with new Gemini 2.5 reasoning model 
The 2.5 flash model is the first hybrid reasoning models by Google

- Track costs of thoughts separately as they are priced differently
 2025-04-18 17:37:05 +05:30
Debanjum
f95173bb0a Support deepthought in research mode with new Grok 3 reasoning model 
Rely on deepthought flag to control reasoning effort of low/high for
the grok model

This is different from the openai reasoning models which support
low/medium/high and for which we use low/medium effort based on the
deepthought flag

Note: grok is accessible over an openai compatible API
 2025-04-18 17:37:05 +05:30
Debanjum
9c70a0f3f5 Support recently released Openai reasoning models 
- Rely on deepthought flag to control reasoning effort
- Generalize Openai reasoning model check for all o- series models
 2025-04-18 17:32:29 +05:30
Debanjum
2f8283935a Warn and drop empty messages when format messages for Anthropic 
Log dropped empty messages to debug this unexpected state.

Related 0eb2d17
 2025-04-18 17:32:29 +05:30
Debanjum
51e19c6199 Simplify KHOJ_DOMAIN states. All production deployments should set it. 
Do not need KHOJ_DOMAIN to be tri-state.
KHOJ_DOMAIN set to empty does not change behavior anymore.

Related 5a3c7b1
 2025-04-18 17:32:29 +05:30
Debanjum
e072530471 Deduplicate images generated using the e2b code tool 
Disregard chart types as not using rich chart rendering
and they are duplicate of chart images that are rendered

Disregard text output associated with generated image files
 2025-04-18 17:32:29 +05:30
RIKIN BRIGHT
dc398d30f8 Add FAQ section to Readme and Troubleshooting Tips to setup GCP Vertex (#1158) 
Added a “Troubleshooting & Tips” section to the GCP Vertex documentation.

This section provides guidance for self-hosted users on common issues
they may encounter when setting up Google Vertex AI integration in Khoj.
Topics covered include permissions, region compatibility, prompt size
limits, API key testing, and secure key management with environment
variables. The goal is to improve the onboarding experience and reduce
setup errors for contributors and self-hosters using Vertex AI models
like Claude and Gemini.

Signed off by: brightally6@gmail.com
 2025-04-15 08:19:44 +05:30
sabaimran
6a30da3e9e Fix default state for tools in the agent settings for the chat sidebar 2025-04-11 11:12:22 -07:00
Debanjum
2470eea421 Release Khoj version 1.40.0 2025-04-11 18:10:56 +05:30
Debanjum
d0a933b072 Add email based rate limiting to email login API endpoint 
Server:
 - Rate limit based on unverified email before creating user
 - Check email address for deliverability before creating user
 - Track rate limit for unverified email in new non-user keyed table

Web app:
 - Show error in login popup to user on failure/throttling
 - Simplify login popup logic by moving magic link handling logic
   into EmailSigninContext instead of passing require props via parent
 2025-04-11 17:49:18 +05:30
Debanjum
fe308c2911 Handle scenario where no valid otps for selected users on admin panel 2025-04-11 17:49:18 +05:30
Debanjum
02a6ce9f14 Upgrade server django dependencies 2025-04-11 17:49:18 +05:30
Debanjum
d84a0abb7f Fix and Improve Chat sidebar and component setup on Web App (#1157) 
- Set chatSidebar prompt, Setting name fields to empty str if value null
- Track if agent modified in chatSidebar to simplify code, fix looping
- Suppress spurious dark mode hydration warnings on the web app
- Set key for chatMessage parent to get UX efficiently updated by react
- Let only root next.js layout handle html, body tags, not child layouts
 2025-04-11 16:12:03 +05:30
Dmitry
50b0b8a6e7 Fix typo in Development documentation (#1159) 2025-04-11 16:05:40 +05:30
Darya
f609a2d050 Fix typos in admin panel documentation (#1156) 
These adjustments should make the text clearer and more accurate.
 2025-04-11 07:00:17 +05:30
Debanjum
2935ea52cf Set chatSidebar prompt, Setting name fields to empty str if value null 
TextArea and Input field values cannot be null.
 2025-04-10 19:59:01 +05:30
Debanjum
aea7b90fec Track if agent modified in chatSidebar to simplify code, fix looping 
Previously the sidebar could recurse on opening chat page (from home?)
due to child modelSelector component updating parent chatSidebar prop
which was passed back down to it in a loop.

The chatSidebar decides if agent has been modified in a single
useEffect and enables the Save button accordingly.
- Track agent modification wrt agent info received from server in
  chatSidebar instead.
- Reduce modelSelector's mandate to just notify
  when the user changes the model.

- Fix to infer, show & update agent state from chat sidebar on web app
  This logic is fragile and convoluted because:
  - the default agent chat model is dynamically determined.
  - need to disambiguate tools not set vs none set vs all set by user
    The default agent's tool selection is stored as undefined to show
    not set scenario, which allows for all tools to be dynamically
    used by  agent.
    But the user can also set no tools or all tools for their agents.
    All 3 scenarios are handled differently.
  - Track tools to be displayed vs tools to be stored
 2025-04-10 19:59:01 +05:30
Debanjum
e9ee9004fb Suppress spurious dark mode hydration warnings on the web app 
This is triggered by mismatch between "dark" class present on server
sent layout but not in client sent layout on initial render.

That mismatch exists because the server applies dark-mode styling
early to avoid FOUC flickering of UX.

Related 43e032e
 2025-04-10 19:59:01 +05:30
Debanjum
9ab5ead3ca Set key for chatMessage parent to get UX efficiently updated by react 
By fixing the no key prop in ChatHistory error on web app
 2025-04-10 19:59:01 +05:30
Debanjum
1ad7314fe6 Let only root next.js layout handle html, body tags, not child layouts 
Remove html, body elements from child page layouts. Let only the root
layout handle it.

Next.js router structure mounts child layouts inside parent layouts,
as defined by their directory hierarchy. So the html, body component
should only be defined in the parent layout.

This avoids the child layout mounting its html, body component within
the actual root layout's existing html, body component.
 2025-04-10 19:59:01 +05:30
Debanjum
33665dee50 Dynamically set default agent chat model to server > user > first chat model 
Previously the chat model associated with the default agent was always
the first chat model populated on the server. This doesn't match
behavior of the rest of the system, where the server chat settings is
preferred over the user chat settings over the first chat model.

This change brings the default agent's chat model in line with the
preference order used in the reset of the system.
 2025-04-10 19:58:01 +05:30
Debanjum
1eb092010c Fix handling unset response_schema being passed to gemini models 
Use of `is_none_or_empty' doesn't work well with classes, which
response_schema can get set to.
 2025-04-10 19:58:01 +05:30
Debanjum
5b248e8515 Fix fallback to default agent if none set for conversation 
Previous change to fallback to default agent was not functional. It
would error out if the conversation agent wasn't set when trying to
get conversation.agent.slug for calling aget_agent_by_slug func
 2025-04-10 19:54:01 +05:30
Debanjum
4012a6372f Fix pgvector docker image pull by using postgres 15 tag 
We were previously relying on an older, unmaintained version of
pgvector docker image, ankane/pgvector.

Moving to new docker image requires selecting from tags based on the
pg major version (14, 15, 16 or 17).

This change uses pg15 tag to resolve image pull.
Note: we use postgres 15 for khoj docker images currently

Fixes #1154
 2025-04-08 18:07:08 +05:30
Debanjum
19618605a5 Upgrade documentation packages 2025-04-07 20:07:10 +05:30
Debanjum
3fc1435cd1 Fix login to local admin panel without need to set KHOJ_DEBUG 
Issue introduced in commit 5a3c7b1.

Usage of KHOJ_DOMAIN
---
KHOJ_DOMAIN is tri-state for local, official and other production deployments:
- If KHOJ_DOMAIN is unset (for local):
  - sets CSRF cookie to localhost
  - adds khoj.dev variants to ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS
  - adds app.khoj.dev variants to CORS origins
- If KHOJ_DOMAIN is set to empty (for official):
  - sets CSRF to khoj.dev
  - adds khoj.dev variants to ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS
  - adds app.khoj.dev variants to CORS origins
- If KHOJ_DOMAIN is set (for other prod deployments):
  - sets CSRF cookie to KHOJ_DOMAIN
  - adds KHOJ_DOMAIN variants to ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS
  - adds KHOJ_DOMAIN variants to CORS origins

Related #1137, #1152

Resolves #1123
 2025-04-07 19:43:58 +05:30
Debanjum
ed70d2254e Suppress spurious RequestAborted ASGI errors on the admin panel 
Unsure why this error triggers on every request to the Django admin
panel these days but all the requests are completing fine and the
client is clearly not aborting the request when the RequestAborted
exception is raised.

Suppress these errors for now via middleware to prevent them from
unnecessarily cluttering up the server logs and confusing folks.

Related #1152
 2025-04-07 19:43:58 +05:30
Debanjum
353a4aa597 Upgrade pgvector to use the new official docker image 
- ankane/pgvector has been deprecated in place of pgvector/pgvector
- use full path to image on docker.io for consistency
 2025-04-07 19:43:58 +05:30
Debanjum
50508d97f9 Disable telemetry only via KHOJ_TELEMETRY_DISABLE environment variable 
Do not unnecessarily overload usage of KHOJ_DEBUG environment variable
 2025-04-07 19:43:58 +05:30
sabaimran
2ae8c62547 Repopulate the client API-key generation section 2025-04-07 06:54:47 -07:00
Debanjum
6872817d41 Remove request to set default chat model during interactive init 
It wasn't being set correctly and seems unnecessary as can switch
to desired chat models via the settings page or chat side pane easily
 2025-04-07 14:38:55 +05:30
Debanjum
0c257c044e Handle unset response_schema being passed to gemini models 2025-04-05 23:12:35 +05:30
Debanjum
c1912f8ca7 Default eval to use 10 iterations for research mode 2025-04-05 10:09:58 +05:30
Debanjum
645c2bc546 Improve Khoj is ready message 2025-04-05 10:09:28 +05:30
Debanjum
6e61ec64a4 Release Khoj version 1.39.0 2025-04-04 20:35:50 +05:30
Debanjum
751215a701 Improve response quality with Gemini. Improve evaluation harness (#1150) 
### Improve Gemini usage
- Allow text tool to give agent ability to terminate research
- Set default context for gemini 2 flash models
  2x context window for small, commercial models to 120K
- Default temperature of Gemini models to 1.0 to reduce repetition

### Improve evaluation harness
- Add more knobs to control eval workflow
  - Allow running eval with any chat model served over an openai compatible api
  - Control random sampling from eval set
  - Auto read web page
- Use embedded postgres instead of postgres server for eval workflow
- Use Gemini 2.0 flash as evaluator. Set seed for evaluator to reduce decision variance
 2025-04-04 20:17:36 +05:30
Debanjum
7f18bc0840 Add default context for gemini 2 flash. 2x it for small, commercial models 
Previously Gemini 2 flash and flash lite were using context window of
10K by default as no defaults were added for it.

Increase default context for small commercial models to 120K from 60K
as cheaper and faster than their pro models equivalents at 60K context.
 2025-04-04 20:11:00 +05:30
Debanjum
47a081c7bd Allow text tool to give agent ability to terminate research 
We'd moved research planner to only use tools in enum of schema. This
enum tool enforcement prevented model from terminating research by
setting tool field to empty.

Fix the issue by adding text tool to research tools enum and tell
model to use that to terminate research and start response instead.
 2025-04-04 20:11:00 +05:30
Debanjum
38dd02afbf Make ordering of fields expected by research planner consistent 
Make research planner consistently select tool before query. As the
model should tune it's query for the selected tool. It got space to
think about tool to use in the scratchpad already.
 2025-04-04 20:11:00 +05:30
Debanjum
443c5a4420 Consistently wrap queries in online search prompt in double quotes 
The queries field name in the first example isn't wrapped in double
quotes, rest are.
 2025-04-04 20:11:00 +05:30
Debanjum
ae8fb6f9ac Default temperature of Gemini models to 1.0 to try avoid repetition 
This is the default temperature for non-thinking gemini models on ai
studio. See if using this alleviates the problem.
 2025-04-04 20:11:00 +05:30
Debanjum
e9928d3c50 Eval more model, control randomization & auto read webpage via workflow 
- Control auto read webpage via eval workflow. Prefix env var with KHOJ_
  Default to false as it is the default that is going to be used in prod
  going forward.

- Set openai api key via input param in manual eval workflow runs
  - Simplify evaluating other chat models available over openai
    compatible api via eval workflow.
  - Mask input api key as secret in workflow.
  - Discard unnecessary null setting of env vars.

- Control randomization of samples in eval workflow.
  If randomization is turned off, it'll take the first SAMPLE_SIZE
  items from the eval dataset instead of a random collection of
  SAMPLE_SIZE items.
 2025-04-04 20:11:00 +05:30
Debanjum
911e1bf981 Use gemini 2.0 flash as evaluator. Set seed for it to reduce eval variance. 
Gemini 2.0 flash model is cheaper and better than Gemini 1.5 pro
 2025-04-04 20:11:00 +05:30
Debanjum
0dcb2544d7 Use embedded postgres instead of postgres server for eval workflow 2025-04-04 20:11:00 +05:30
sabaimran
abfdf7b1fb Temporarily remove the OTP feature on the settings page while we fix our number 2025-04-03 19:41:18 -05:00
Artem Yurchenko
1ef8c37c3a Implement better template for feature request issue (#1132) 
This PR implements a new feature request template with a few UX/UI improvements.

Key changes:
- Use of GitHub forms.
- Provide note info for a submitter about feature request submitting rules.
- Adds a few handy fields like "Describe the feature" or "Use Case"

Overall, with a template like this feature requests will be more structured and meaningful.
 2025-04-03 15:41:04 +05:30
Debanjum
17997d26c5 Make generated code block extractor less strict to improve code tool 2025-04-03 14:07:15 +05:30
Debanjum
66e9ddb6be Support OpenAI (API compatible) models and Firecrawl in eval workflow 2025-04-03 14:03:29 +05:30
Debanjum
ef5b19479f Improve setup of openai compatible speech to text, image models 
Only setup speech to text and text to image models served via openai
compatible APIs when explicitly specified during initialization.

This avoids setup of whisper and dalle when an openai compatible API
is being setup instead of the openai API itself.
 2025-04-03 13:44:19 +05:30
Debanjum
c2b9006a7a Improve Gemini Response Reliability (#1148) 
- Specify min, max number of list items expected in AI response via JSON schema enforcement. Used by Gemini models
- Warn and drop invalid/empty messages when format messages for Gemini models
- Make Gemini response adhere to the order of the schema property definitions
- Improve agent creation safety checker by using response schema, better prompt
 2025-04-03 13:42:35 +05:30
Debanjum
f77e871cc8 Improve agent creation safety checker with response schema, better prompt 2025-04-03 02:49:40 +05:30
Debanjum
aab010723c Make Gemini response adhere to the order of the schema property definitions 
Without explicitly using the property ordering field, gemini returns
responses in alphabetically sorted property order.

We want the model to respect the schema property definition order.
This ensures control during development to maintain response quality.

For example in CoT make it fill scratchpad before answers.
 2025-04-03 01:19:48 +05:30
Debanjum
ae9ca58ab9 Specify min, max items expected in ai response via schema enforcement 
Require at least 1 item in lists. Otherwise gemini flash will
sometimes return an empty list. For chat actors where max items is
known, set that as well.

OpenAI API does not support specifying min, max items in response
schema lists, so drop those properties when response schema is
passed. Add other enforcements to response schema to comply with
response schema format expected by OpenAI API.
 2025-04-03 01:19:39 +05:30
Debanjum
0eb2d17771 Warn and drop invalid messages when format messages for gemini 
Previously we were setting message content part with empty text. This
results in error from Gemini API. Warn and drop such messages instead.

Log empty message content found during construction to root-cause the
issue but allow Khoj to respond without the offending messages in
context for call to Gemini API.
 2025-04-03 01:11:22 +05:30
sabaimran
dd957bedd3 Remove pgserver in repo from git tracking 2025-03-31 15:07:24 -05:00
sabaimran
d53b740197 Improve online search and allow server to skip auto webpage read 2025-03-31 13:52:48 -05:00
Debanjum
177560655d Fix and Improve Online Search and Webpage Read (#1147) 
New
- Support Firecrawl as a online search provider

Improve
- Fallback to other enabled online search providers on failure
- Speed up online search with Jina by excluding webpage content in search results

Fix
- Fix Jina webpage reader. Improve it to include generated alt text to each image on webpage
- Truncate online query to Serper if query exceeds max supported length
 2025-04-01 00:09:46 +05:30
Debanjum
d62dd4ef61 Support Firecrawl as a online search provider 2025-03-31 17:06:00 +05:30
Debanjum
3939e995e4 Fallback to enabled, lower priority online search providers on error 
Make serper.dev higher priority than official google serp api because
it provides more detailed results with knowledge cards etc.
 2025-03-31 17:05:44 +05:30
Debanjum
9b7442f28f Truncate online query to Serper if query exceeds max supported length 
Previously query to serper with longer than max supported would throw
error instead of returning at least some results.

Truncating the onlien search query to serper to max supported length
mitigates that issue.
 2025-03-31 17:01:42 +05:30
Debanjum
db7eba56f6 Fix webpage read and improve web search with Jina 
- Improve webpage read to include image alt text
- Improve Jina webpage search to not include each page content
- Use POST instead of GET for web search, webpage read with Jina
 2025-03-31 17:01:42 +05:30
Debanjum
db68372b81 Update code sandbox prompts to allow network access when using E2B 
Tell Khoj code writing chat actor that it has access to the network
and can use the python requests library in the E2B code sandbox.
 2025-03-31 15:33:47 +05:30
Debanjum
5b8c2989d6 Add hover text on button to unshare a conversation on web app 2025-03-31 15:32:43 +05:30
Debanjum
85d627ceb0 Simplify docs to self-host with pip since can use embedded DB now 
Remove postgres setup instructions from self host with pip docs. It is
unnecessary if embedded postgres DB works on the operating system.
 2025-03-30 00:17:24 +05:30
Debanjum
713ba06a8d Release Khoj version 1.38.0 2025-03-29 18:30:06 +05:30
Debanjum
e9132d4fee Support attaching programming language file types to web app for chat 2025-03-29 01:22:35 +05:30
Debanjum
bdb6e33108 Install pgserver only when `pip install khoj[local]' is enabled 
This avoids installing pgserver on linux arm64 docker builds, which it
doesn't currently support and isn't required to support as Khoj docker
images can use standard postgres server made available via our
docker-compose.yml
 2025-03-29 00:27:19 +05:30
Debanjum
5ee513707e Use embedded postgres db to simplify self-hosted setup (#1141) 
Use pgserver python package as an embedded postgres db,
installed directly as a khoj python package dependency.

This significantly simplifies self-hosting with just a `pip install khoj'. 
No need to also install postgres separately.

Still use standard postgres server for multi-user, production use-cases.
 2025-03-29 00:03:55 +05:30
Debanjum
56b63f95ea Suggest Google image gen model, new Anthropic chat models on first run 
- Update default anthropic chat models to latest good models.

- Now that Google supports a good text to image model. Suggest adding
  that if Google AI API is setup on first run.
 2025-03-28 23:07:17 +05:30
Ikko Eltociear Ashimine
1e34de69e9 Fix spelling in Automations Docs (#1140) 
Recieve -> Receive
 2025-03-28 23:07:06 +05:30
Debanjum
72986c905a Fix default agent creation to allow chat on first run 
Previously agent slug was not considered on create even when passed
explicitly in agent creation step.

This made the default agent slug different until next run when it was
updated after creation. And didn't allow chat to work on first run

The fix to use the agent slug when explicitly passed allows users to
chat on first run.
 2025-03-28 22:49:00 +05:30
Debanjum
03de2803f0 Fallback to default agent for chat when unset in get conversation API 2025-03-28 00:56:18 +05:30
Debanjum
a387f638cd Enforce json schema on more chat actors to improve schema compliance 
Including infer webpage urls, gemini documents search, pick default
mode tools chat actors
 2025-03-28 00:56:18 +05:30
Debanjum
ccd9de7792 Improve safety settings for Gemini chat models 
- Align remaining harm categories to only refuse in high harm
  scenarios as well
- Handle response for new "negligible" harm probability as well
 2025-03-28 00:56:18 +05:30
Debanjum
2ec5cf3ae7 Normalize type of chat messages arg sent to Anthropic completion funcs 
Previously messages got Anthropic specific formatting done before
being passed to Anthropic (chat) completion functions.

Move the code to format messages of type list[ChatMessage] into Anthropic
specific format down to the Anthropic (chat) completion functions.

This allows the rest of the functionality like prompt tracing to work
with normalized list[ChatMesssage] type of chat messages across AI API
providers
 2025-03-26 18:24:17 +05:30
Debanjum
4085c9b991 Fix infer webpage url step actor to request upto specified max urls 
Previously we'd always request up to 3 webpage url via the prompt but
read only one of the requested webpage url.

This would degrade quality of research and default mode. As model may
request reading upto 3 webpage links but get only one of the requested
webpages read.

This change passes the number of webpages to read down to the AI model
dynamically via the updated prompt. So number of webpages requested to
be read should mostly be same as number of webpages actually read.

Note: For now, the max webpages to read is kept same as before at 1.
 2025-03-26 18:24:17 +05:30
Debanjum
c337c53452 Fix to use agent chat model for research model planning 
Previously the research mode planner ignored the current agent or
conversation specific chat model the user was chatting with. Only the
server chat settings, user default chat model, first created chat model
were considered to decide the planner chat model.

This change considers the agent chat model to be used for the planner
as well. The actual chat model picked is decided by the existing
prioritization of server > agent > user > first chat model.
 2025-03-25 18:31:55 +05:30
Debanjum
df090e5226 Enable unsharing of a public conversation (#1135) 
This change enables the creator of a shared conversation to stop sharing the conversation publicly.

### Details
1. Create an API endpoint to enable the owner of the shared conversation to unshare it
2. Unshare a public conversations from the title pane of the public conversation on the web app
 2025-03-25 14:24:01 +05:30
Debanjum
9dfa7757c5 Unshare public conversations from the title pane on web app 
Only show the unshare button on public conversations created by the
currently logged in user. Otherwise hide the button

Set conversation.isOwner = true only if currently logged in user
shared the current conversation.

This isOwner information is passed by the get shared conversation API
endpoint
 2025-03-25 14:05:29 +05:30
Debanjum
d9c758bcd2 Create API endpoint to unshare a public conversation 
Pass isOwner field from the get shared conversation API endpoint if
the currently authenticated user created the requested public
conversation
 2025-03-25 14:05:29 +05:30
Debanjum
e3f6d241dd Normalize chat messages sent to gemini funcs to work with prompt tracer 
Previously messages passed to gemini (chat) completion functions
got a little of Gemini specific formatting mixed in.

These functions expect a message of type list[ChatMessage] to work
with prompt tracer etc.

Move the code to format messages of type list[ChatMessage] into gemini
specific format down to the gemini (chat) completion functions.

This allows the rest of the functionality like prompt tracing to work
with normalize list[ChatMesssage] type of chat messages across
providers
 2025-03-25 14:04:16 +05:30
Debanjum
7976aa30f8 Terminate research if query or tool is empty 2025-03-25 14:04:16 +05:30
Debanjum
39aa48738f Set effort for openai reasoning models to pick tool in research mode 
This is analogous to how we enable extended thinking for claude models
in research mode.

Default to medium effort irrespective of deepthought for openai
reasoning models as high effort is currently flaky with regular
timeouts and low effort isn't great.
 2025-03-25 14:04:16 +05:30
Debanjum
b4929905b2 Add costs of ai prompt cache read, write. Use for calls to Anthropic 2025-03-25 14:04:16 +05:30
Debanjum
d4b0ef5e93 Fix ability to disable code and internet providers in eval workflow 
Sets env vars to empty if condition not met so:
- Terrarium (not e2b) used as code sandbox on release triggered eval
- Internet turned off for math500 eval
 2025-03-25 14:04:16 +05:30
sabaimran
a8285deed7 Release Khoj version 1.37.2 2025-03-23 11:38:25 -07:00
sabaimran
b7ac8771de Update a few pieces of documentation around data sources. 2025-03-23 11:36:20 -07:00
sabaimran
12e7409da9 Release Khoj version 1.37.1 2025-03-23 11:10:34 -07:00
sabaimran
985f1672ed Remove eval lists from git tracking 2025-03-23 10:59:32 -07:00
Debanjum
d1df9586ca Standardize AI model response temperature across provider specific ranges 
- Anthropic expects a 0-1 range. Gemini & OpenAI expect a 0-2 range
- Anneal temperature to explore reasoning trajectories but respond factually
- Default send_message_to_model and extract_question temps to the same
 2025-03-23 18:09:22 +05:30
Debanjum
55ae0eda7a Upgrade package dependencies nextjs for web app and torch on server 2025-03-23 17:10:40 +05:30
Debanjum
8409e64ff0 Clean AI model API providers documentation 2025-03-23 16:26:34 +05:30
Debanjum
86a51d84ca Access Claude and Gemini via GCP Vertex AI (#1134) 
Support accessing Claude and Gemini AI models via Vertex AI on Google Cloud. 

See the documentation at docs.khoj.dev for setup details
 2025-03-23 16:26:02 +05:30
Debanjum
16ffebf765 Document how to configure using AI models via GCP Vertex AI 2025-03-23 16:12:46 +05:30
Debanjum
7153d27528 Cache Google AI API client for reuse 2025-03-23 16:12:46 +05:30
Debanjum
da33c7d83c Support access to Gemini models via GCP Vertex AI 2025-03-23 16:12:46 +05:30
Debanjum
603c4bf2df Support access to Anthropic models via GCP Vertex AI 
Enable configuring a Khoj AI model API for Vertex AI using GCP credentials.

Specifically use the api key & api base url fields of the AI Model API
associated with the current chat model to extract gcp region, gcp
project id & credentials. This helps create a AnthropicVertex client.

The api key field should contain the GCP service account keyfile as a
base64 encoded string.

The api base url field should be of the form
`https://{MODEL_GCP_REGION}-aiplatform.googleapis.com/v1/projects/{YOUR_GCP_PROJECT_ID}`

Accepting GCP credentials via the AI model API makes it easy to use
across local and cloud environments. As it bypasses the need for a
separate service account key file on the Khoj server.
 2025-03-23 16:12:46 +05:30
Debanjum
8bebcd5f81 Support longer API key field in DB to store GCP service account keyfile 2025-03-23 14:55:50 +05:30
Debanjum
f2b438145f Upgrade sentence-transformers. Avoid transformers v4.50.0 as problematic 
- The 3.4.1 release of sentence tranformer fixes offline load latency
  of sentence transformer models (and Khoj) by avoiding call to HF

- The 4.50.0 release of transformers is resulting in
  jax error (unexpected keyword argument 'flatten_with_keys') on load.
 2025-03-23 09:02:57 +05:30
Debanjum
510cbed61c Make google auth package dependency explicit to simplify code 
Previously google auth library was explicitly installed only for the
cloud variant of Khoj to minimize packages installed for non
production use-cases.

But it was being implicitly installed as a dependency of an explicit
package in the default installation anyway.

Making the dependency on google auth package explicit simplifies
the conditional import of google auth in code while not incurring any
additional cost in terms of space or complexity.
 2025-03-23 09:02:57 +05:30
Debanjum
5fff05add3 Set seed for Google Gemini models using KHOJ_LLM_SEED env variable 
This env var was already being used to set seed for OpenAI and Offline
models
 2025-03-22 08:59:31 +05:30
Debanjum
6cc5a10b09 Disable SimpleQA eval on release as saturated & low signal for usecase 
Reaching >94% in research mode on SimpleQA. When answers can be
researched online, it becomes too easy. And the FRAMES eval does a
more thorough job of evaluating that use-case anyway.
 2025-03-22 08:05:12 +05:30
Debanjum
45015dae27 Limit to json enforcement via json object with DeepInfra hosted models 
DeepInfra based models do not seem to support json schema. See
https://deepinfra.com/docs/advanced/json_mode for reference
 2025-03-22 08:04:09 +05:30
Debanjum
dc473015fe Set default model, sandbox to display in eval workflow summary on release 2025-03-20 14:44:56 +05:30
Debanjum
80d864ada7 Release Khoj version 1.37.0 2025-03-20 14:06:57 +05:30
Debanjum
0c53106b30 Fix passing inline images to vision models 
- Fix regression: Inline images were not getting passed to the AI
  models since #992
- Format inline images passed to Gemini models correctly
- Format inline images passed to Anthropic models correctly

Verified vision working with inline and url images for OpenAI,
Anthropic and Gemini models.

Resolves #1112
 2025-03-20 13:22:46 +05:30
Debanjum
1ce1d2f5ab Deduplicate, clean code for S3 images uploads 2025-03-20 12:30:07 +05:30
Debanjum
f15a95dccf Show Khoj agent in agent dropdown by default on mobile in web app home 
Previously on slow connection you'd see the agent dropdown flicker
from undefined to Khoj default agent on phones and other thin screens.

This is unnecessary and jarring. Populate with default agent to remove
this issue
 2025-03-20 12:27:52 +05:30
Debanjum
9a0b126f12 Allow chat input on web app while Khoj responds to speed interactions 
Previously the chat input area didn't allow inputting text while Khoj is
researching and generating response.

This change allows the user to add their next text while Khoj
responds. This should speed up interaction cycles as user can have
their next query ready to send when Khoj finishes its response.
 2025-03-19 23:08:22 +05:30
Debanjum
e68428dd24 Support enforcing json schema in supported AI model APIs (#1133) 
- Trigger
  Gemini 2.0 Flash doesn't always follow JSON schema in research prompt

- Details
  - Use json schema to enforce generate online queries format
  - Use json schema to enforce research mode tool pick format
  - Support constraining Gemini model output to specified response schema
  - Support constraining OpenAI model output to specified response schema
  - Only enforce json output in supported AI model APIs
  - Simplify OpenAI reasoning model specific arguments to OpenAI API
 2025-03-19 22:59:23 +05:30
Debanjum
a5627ef787 Use json schema to enforce generate online queries format 2025-03-19 22:32:53 +05:30
Debanjum
2c53eb9de1 Use json schema to enforce research mode tool pick format 2025-03-19 22:32:53 +05:30
Debanjum
6980014838 Support constraining Gemini model output to specified response schema 
If the response_schema argument is passed to
send_message_to_model_wrapper it is used to constrain output by Gemini
models
 2025-03-19 22:32:53 +05:30
Debanjum
ac4b36b9fd Support constraining OpenAI model output to specified response schema 2025-03-19 22:32:52 +05:30
Debanjum
4a4d225455 Only enforce json output in supported AI model APIs 
Deepseek reasoner does not support json object or schema via deepseek API
Azure Ai API does not support json schema

Resolves #1126
 2025-03-19 22:32:11 +05:30
Debanjum
d74c3a1db4 Simplify OpenAI reasoning model specific arguments to OpenAI API 
Previously OpenAI reasoning models didn't support stream_options and
response_format

Add reasoning_effort arg for calls to OpenAI reasoning models via API.
Right now it defaults to medium but can be changed to low or high
 2025-03-19 21:12:02 +05:30
Debanjum
9b6d626a09 Fix to store e2b code execution text output file content as string 
Previously was encoding E2B code execution text output content as b64.

This was breaking
- The AI model's ability to see the content of the file
- Downloading the output text file with appropriately encoded content

Issue created when adding E2B code sandbox in #1120
 2025-03-19 20:09:41 +05:30
Artem Yurchenko
a7e261a191 Implement better bug issue template (#1129) 
* Implement better bug issue template
* Fix IDs in new bug issue template
* Reduce, reorder and improve field descriptions in the bug issue template

---------

Co-authored-by: Debanjum <debanjum@gmail.com>
 2025-03-18 20:53:57 +05:30
Debanjum
931f555cf8 Configure max allowed iterations in research mode via env var 2025-03-18 18:15:50 +05:30
Debanjum
2ab8e711d3 Fix Gemini models to output valid json when configured 2025-03-18 17:02:45 +05:30
sabaimran
ce60cb9779 Remove max-w 80vw, which was smushing AI responses 2025-03-13 13:44:05 -07:00
sabaimran
a3c4347c11 Add a one-click action to export all conversations. Add a self-service delete account action to the settings page 2025-03-12 23:54:02 -07:00
Debanjum
79816d2b9b Upgrade package dependencies of server, clients and docs 2025-03-12 00:22:08 +05:30
Debanjum
7bb6facdea Add support for Google Imagen AI models for image generation 
Use the new Google GenAI client to generate images with Imagen
 2025-03-11 23:39:46 +05:30
Debanjum
bd06fcd9be Stop using old google generativeai package to raise, catch exceptions 2025-03-11 23:39:46 +05:30
Debanjum
bdfa6400ef Upgrade to new Gemini package to interface with Google AI 2025-03-11 22:18:07 +05:30
Debanjum
2790ba3121 Update default temperature for calls to Gemini models to 0.6 from 0.2 
This aligns with default temperature used by google ai studio and
may reduce loops and repetitions
 2025-03-11 21:28:04 +05:30
Debanjum
50f71be03d Support Claude 3.7 and use its extended thinking in research mode 
Claude 3.7 Sonnet is Anthropic's first reasoning model. It provides a
single model/api capable of standard and extended thinking. Utilize
the extended thinking in Khoj's research mode.

Increase default max output tokens to 8K for Anthropic models.
 2025-03-11 21:27:59 +05:30
Debanjum
69048a859f Fix E2B tool description prompt to mention plotly package available 2025-03-11 02:20:06 +05:30
Debanjum
9751adb1a2 Improve Code Tool, Sandbox and Eval (#1120) 
# Improve Code Tool, Sandbox
  - Improve code gen chat actor to output code in inline md code blocks
  - Stop code sandbox on request timeout to allow sandbox process restarts
  - Use tenacity retry decorator to retry executing code in sandbox
  - Add retry logic to code execution and add health check to sandbox container
  - Add E2B as an optional code sandbox provider

# Improve Gemini Chat Models
  - Default to non-zero temperature for all queries to Gemini models
  - Default to Gemini 2.0 flash instead of 1.5 flash on setup
  - Set default chat model to KHOJ_CHAT_MODEL env var if set
 2025-03-09 18:49:59 +05:30
Debanjum
c133d11556 Improvements based on code feedback 2025-03-09 18:23:30 +05:30
Debanjum
94ca458639 Set default chat model to KHOJ_CHAT_MODEL env var if set 
Simplify code log to set default_use_model during init for readability
 2025-03-09 18:23:30 +05:30
Debanjum
7b2d0fdddc Improve code gen chat actor to output code in inline md code blocks 
Simplify code gen chat actor to improve correct code gen success,
especially for smaller models & models with limited json mode support

Allow specify code blocks inline with reasoning to try improve
code quality

Infer input files based on user file paths referenced in code.
 2025-03-09 18:23:30 +05:30
Debanjum
8305fddb14 Default to non-zero temperature for all queries to Gemini models. 
It may mitigate the intermittent invalid json output issues. Model
maybe going into repetition loops, non-zero temp may avoid that.
 2025-03-09 18:23:30 +05:30
Debanjum
45fb85f1df Add E2B as an optional code sandbox provider 
- Specify E2B api key and template to use via env variables
- Try load, use e2b library when E2B api key set
- Fallback to try use terrarium sandbox otherwise
- Enable more python packages in e2b sandbox like rdkit via custom e2b template

- Use Async E2B Sandbox
- Parallelize file IO with sandbox
- Add documentation on how to enable E2B as code sandbox instead of Terrarium
 2025-03-09 18:23:30 +05:30
Debanjum
b4183c7333 Default to gemini 2.0 flash instead of 1.5 flash on Gemini setup 
Add price of gemini 2.0 flash for cost calculations
 2025-03-07 13:48:15 +05:30
Debanjum
701a7be291 Stop code sandbox on request timeout to allow sandbox process restarts 2025-03-07 13:48:15 +05:30
Debanjum
ecc2f79571 Use tenacity retry decorator to retry executing code in sandbox 2025-03-07 13:48:15 +05:30
sabaimran
4a28714a08 Add retry logic to code execution and add health check to sandbox container 2025-03-07 13:48:15 +05:30
Debanjum
f13bdc5135 Log eval run progress percentage for orientation 2025-03-07 13:48:15 +05:30
Debanjum
bbe1b63361 Improve Obsidian Sync for Large Vaults (#1078) 
- Batch sync files by size to try not exceed API request payload size limits
- Fix force sync of large vaults from Obsidian
- Add API endpoint to delete all indexed files by file type
- Fix to also delete file objects when call DELETE content source API
 2025-03-07 13:47:21 +05:30
Debanjum
043de068ff Fix force sync of large vaults from Obsidian 
Previously if you tried to force sync a vault with more than 1000
files it would only end up keeping the last batch because the PUT API
call would delete all previous entries.

This change calls DELETE for all previously indexed data first, followed by
a PATCH to index current vault on a force sync (regenerate) request.
This ensures that files from previous batches are not deleted.
 2025-03-07 13:34:48 +05:30
Debanjum
86fa528a73 Add API endpoint to delete all indexed files by file type 2025-03-07 13:28:53 +05:30
Debanjum
b692e690b4 Rename and fix the delete content source API endpoint 
- Delete file objects on deleting content by source via API
  Previously only entries were deleted, not the associated file objects
- Add new db adapter to delete multiple file objects (by name)
 2025-03-07 13:28:53 +05:30
Debanjum
29403551b2 Batch sync files by size to not exceed API request payload size limits 
This may help mitigate the issue #970
 2025-03-04 09:31:48 +05:30
Debanjum
0ddb6a38b8 Update Khoj docs to mark LMStudio as not supported anymore 
They seem to have deprecated json mode in their openai compatible API
which Khoj uses extensively.
 2025-02-18 21:33:03 +05:30
sabaimran
de550e5ca7 re-enable markdown formatting when chatting with o3 2025-02-18 21:12:03 +05:30
sabaimran
0016fe06c9 Release Khoj version 1.36.6 2025-02-18 18:54:13 +05:30
sabaimran
7089ea1cf4 Remove experimental parenthesis from research mode ✁ 2025-02-18 08:59:12 +05:30
Debanjum
5a3c7b145a Decouple Django CSRF, ALLOWED_HOST settings for more complex setups 
- Set KHOJ_ALLOWED_DOMAIN to the domain that Khoj is accessible on
  from the host machine. This can be the internal i.p or domain of the
  host machine.

  It can be used by your load balancer/reverse_proxy to access Khoj.
  For example, if the load balancer service is in the khoj docker
  network, KHOJ_DOMAIN will be `server' (i.e service name).

- Set KHOJ_DOMAIN to your externally accessible DOMAIN or I.P to avoid
  CSRF trusted origin or unset cookie issue when trying to access the
  khoj admin panel.

Resolves #1114
 2025-02-17 15:47:35 +05:30
Debanjum
bb0828b887 Only show notes tool option to llm for selection when user has documents 2025-02-17 15:23:26 +05:30
Debanjum
5dfb59e1ee Show more references in teaser ref section of chat response on web app 2025-02-15 14:11:17 +05:30
sabaimran
b6e745336b Add s short description to explain what the create agent button does 2025-02-13 14:08:50 -08:00
sabaimran
848a91313d Move the create agent button to the bottom of the sidebar and fix experience when resetting settings 2025-02-12 19:24:02 -08:00
sabaimran
d0d30ace06 Add feature to create a custom agent directly from the side panel with currently configured settings 
- Also, when in not subscribed state, fallback to the default model when chatting with an agent
- With conversion, create a brand new agent from inside the chat view that can be managed separately
 2025-02-12 18:24:41 -08:00
sabaimran
5d6eca4c22 Fix automation retrieval validity check 2025-02-11 19:17:37 -08:00
sabaimran
51952364ab Release Khoj version 1.36.5 2025-02-11 13:23:30 -08:00
sabaimran
0211151570 Disable auto-setup of offline models if in non-interactive offline mode 2025-02-10 18:48:00 -08:00
sabaimran
589b047d90 Simplify agent picker selection in homepage mobile view 2025-02-10 15:05:30 -08:00
sabaimran
427ec061b4 Add auto redirect on delete of current conversation 2025-02-09 11:38:44 -08:00
sabaimran
bbb5fd667a update messaging in the welcome email 2025-02-07 15:10:59 -08:00
sabaimran
ff6cb80c84 Release Khoj version 1.36.4 2025-02-06 16:50:50 -08:00
sabaimran
031bccb628 Fix awkward padding in chat window 2025-02-06 16:31:25 -08:00
sabaimran
43e032e25a Improve handling of dark mode theme in order to avoid jitter when loading new page 2025-02-06 16:17:58 -08:00
sabaimran
2e01a95cf1 improve system prompt for generating mermaid.js diagrams 2025-02-06 15:05:30 -08:00
sabaimran
a2af6bea8e Release Khoj version 1.36.3 2025-02-04 13:23:41 -08:00
sabaimran
0d10c5fb02 Improve default selection of models to avoid infinite loops 2025-02-04 11:36:41 -08:00
sabaimran
24b1dd3bff Release Khoj version 1.36.2 2025-02-03 20:22:49 -08:00
sabaimran
4409a58794 set initial model of default state 2025-02-03 18:17:21 -08:00
sabaimran
51874c25d5 Prevent infinite loops in model selection logic by configuring an initial model state 2025-02-03 18:11:01 -08:00
sabaimran
489fa71143 Update the width for rending all conversation sessions 2025-02-03 15:49:43 -08:00
sabaimran
b354a37dcd Release Khoj version 1.36.1 2025-02-02 21:55:59 -08:00
sabaimran
61e48d686e Let file context buttons route to search page instead of settings for upload/manage 2025-02-02 12:26:13 -08:00
sabaimran
b4c467cd11 Remove shadows from reference panel trigger icons 2025-02-02 12:23:19 -08:00
sabaimran
a3d75e5241 When in mobile view, don't use the hover card in the model selector 2025-02-02 12:22:45 -08:00
sabaimran
4f79abb429 Release Khoj version 1.36.0 2025-02-02 08:39:21 -08:00
sabaimran
60e6913494 Merge pull request #1094 from khoj-ai/features/add-chat-controls 
Make it easier to determine which model you're chatting with, and to effortlessly update said model from within a given chat. 

In this change, we introduce a side bar that allows users to quickly change their chat model, tools, custom instructions, and file filters, directly within the chat view. This removes the need for setting up custom agents for simple instructions and mitigates the requirement to go to the settings page to verify the chat model in action.

The settings page will still configure a per-user *default*, but the sidebar will allow for greater customization based on the needs of a conversation.

We also extend the chat model to include more attributes that help users make decisions about model selection, including `strengths` and `description`. This can help people quickly understand which model might work best for their use case.
 2025-02-01 14:35:47 -08:00
sabaimran
c558bbfd44 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-chat-controls 2025-02-01 14:25:58 -08:00
sabaimran
08bc1d3bcb Improve sizing / spacing of side bar 2025-02-01 14:25:45 -08:00
sabaimran
f3b2580649 Add back hover state with collapsed references 2025-02-01 14:03:17 -08:00
sabaimran
0645af9b16 Use the agent CM as a backup chat model when available / necessary. Remove automation as agent option. 2025-02-01 13:06:42 -08:00
Debanjum
f2eba667fc Fallback to schedule automation in UTC timezone if unset 
- Handle null automation ids in calls to get_automation function
 2025-02-01 19:12:51 +05:30
Debanjum
0bfa7c1c45 Add support for o3 mini model by openai 2025-02-01 02:51:13 +05:30
sabaimran
641f1bcd91 Only open the side bar automatically when there is no chat history && no pending messages. 2025-01-30 16:07:27 -08:00
sabaimran
b111a9d6c6 Release Khoj version 1.35.3 2025-01-30 11:48:33 -08:00
Yuto SASAKI
018bc718fc Fix typo in docs for Chat Model Type for Google Gemini setup (#1098) 2025-01-30 03:40:37 -08:00
sabaimran
b73f446713 Fallback to show raw outputted diagram if fails rendering 2025-01-29 21:50:28 -08:00
sabaimran
98e3f5162a Show generated diagram raw code if fails rendering 2025-01-29 21:49:33 -08:00
sabaimran
b5f99dd103 Rename custom instructions -> instructions 2025-01-29 15:06:22 -08:00
sabaimran
0ff33d4347 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-chat-controls 2025-01-29 14:58:15 -08:00
sabaimran
c3cb6086e0 Add list typing to the updated_messages temporary variable 2025-01-29 14:19:57 -08:00
sabaimran
5ea056f03e Add custom handling logic when speaking with deepseak reasoner 2025-01-29 14:11:27 -08:00
sabaimran
d640299edc use is_active property for determine user subscription status 2025-01-29 14:10:59 -08:00
sabaimran
e2bfd4ac0f Change name of teams section to teams 2025-01-29 12:52:10 -08:00
sabaimran
58f77edcad Make conversation sessions rounded-lg 2025-01-29 12:47:36 -08:00
sabaimran
0b2305d8f2 Add an animation to opening and closing the thought process 2025-01-29 12:47:14 -08:00
sabaimran
67b2e9c194 Increase subscribed total entries size to 500 MB 2025-01-29 09:06:35 -08:00
sabaimran
01faae0299 Simplify the train of thought UI 2025-01-28 22:00:09 -08:00
sabaimran
59f0873232 Rename train of thought button 2025-01-28 21:32:21 -08:00
sabaimran
272764d734 Simplify the chat response / user message bubbles 2025-01-28 21:17:56 -08:00
sabaimran
b61226779e Simplify references section with icons in chat message 2025-01-28 18:13:26 -08:00
sabaimran
58879693f3 Simplify nav menu and add a teams section 2025-01-28 18:12:50 -08:00
sabaimran
e076ebd133 Make tools section of sidebar a popover to prevent increasing height 2025-01-28 18:12:13 -08:00
sabaimran
ee3ae18f55 add code, remove summarize from agent tools 2025-01-28 18:11:25 -08:00
sabaimran
ba7d53c737 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-chat-controls 2025-01-28 13:01:06 -08:00
Debanjum
af49375884 Only show confusing fallback tokenizer used logs in high verbosity mode 2025-01-25 18:35:14 +07:00
Debanjum
aebdd174ec Make title of PeopleAlsoAsk section of online results optional 2025-01-25 18:35:14 +07:00
sabaimran
43c9ec260d Allow agent personality to be nullable, in which case the default prompt will be used. 2025-01-24 08:24:53 -08:00
sabaimran
2b9a61c987 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-chat-controls 2025-01-23 11:43:51 -08:00
sabaimran
a3b5ec4737 Release Khoj version 1.35.2 2025-01-22 21:42:14 -08:00
sabaimran
938ef0a27b Update DB migration for better memory and speed efficiency 2025-01-22 20:48:37 -08:00
sabaimran
9fc825d7a6 Release Khoj version 1.35.1 2025-01-22 19:51:28 -08:00
sabaimran
5a3a897080 Temporarily move logic to associate entry and fileobject objects into the management command, out of automatic migrations 2025-01-22 19:50:22 -08:00
sabaimran
fd90842d38 Bump postgresql server dev version to 16 for latest ubuntu 2025-01-22 19:07:54 -08:00
sabaimran
1a0923538e Release Khoj version 1.35.0 2025-01-22 19:03:25 -08:00
sabaimran
dc6e9e8667 Skip showing hidden agents in the all conversations agent filter 2025-01-21 12:43:34 -08:00
sabaimran
c1b0a9f8d4 Fix sync to async issue when getting default chat model in hidden agent configuration API 2025-01-21 12:06:09 -08:00
sabaimran
e518626027 Add typing to empty list of operations 2025-01-21 11:56:52 -08:00
sabaimran
e3e93e091d automatically open the side bar when a new chat is created with the default agent. 2025-01-21 11:56:06 -08:00
sabaimran
c43079cb21 Add merge migration and add a new button for new convo in sidebar 2025-01-21 11:01:08 -08:00
sabaimran
5a36360408 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-chat-controls 2025-01-21 10:32:47 -08:00
sabaimran
f96e5cd047 Allow file filter dropdown to pop up automatically when typing "file:" 2025-01-21 08:30:40 -08:00
sabaimran
6d1b16901d Fix the naming of framework > file 2025-01-21 07:24:43 -08:00
Debanjum
7fad1f43f6 Fix text quoting and format web app search page with prettier 2025-01-21 18:18:44 +07:00
sabaimran
8022f040d2 Fix spacing of pagination buttons 2025-01-20 17:23:14 -08:00
sabaimran
3b381a5fe8 Improve default state when no documents are found yet 2025-01-20 17:19:15 -08:00
sabaimran
e0dcd11f34 Allow browsing and discovery of knowlege base in the search page #1073 
Currently, it's rather opaque and difficult to substantially browse through the uploaded knowledge base. Effectively, you can only do this through the small file modal in the settings page. 

Update to include all indexed files in the search page for viewing & deletion. Function to delete all files is still in the settings page.

Add a migration that associates file objects with `entry`s using a foreign key. Add a migration command that deletes dangling fileobjects.
 2025-01-20 16:14:25 -08:00
sabaimran
4b35bee365 Merge branch 'master' of github.com:khoj-ai/khoj into HEAD 2025-01-20 15:35:19 -08:00
sabaimran
8ad60f53d3 Remove export from filefiltercombo box 2025-01-20 15:26:32 -08:00
sabaimran
9f18d6494f Add a file filter combobox for easier file filter selection 2025-01-20 14:39:29 -08:00
sabaimran
d36f235da5 Rename API to get all files, minor UI updates 2025-01-20 13:30:10 -08:00
sabaimran
849b7c7af6 Refresh current page after file deleted 2025-01-20 13:12:37 -08:00
sabaimran
edfed2d571 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-a-knowledge-base-page 2025-01-20 13:11:45 -08:00
sabaimran
b661d2cba7 Merge pull request #1030 from Yash-1511/feat/autocomplete-file-query 
## Description
Added file path autocompletion to enhance the search experience in Khoj. Users can now easily search for specific files by typing "file:" followed by the file name, with real-time suggestions appearing as they type.

## Changes
- Added new API endpoint `/api/file-suggestions` to get file path suggestions
- Enhanced search UI with dropdown suggestions for file paths
- Implemented debounced search to optimize API calls
- Added keyboard (Enter) and mouse click support for selecting suggestions

## Features
- Type "file:" to trigger file path suggestions
- Real-time filtering of suggestions as you type
- Top 10 alphabetically sorted suggestions
- Case-insensitive matching
- Keyboard and mouse interaction support
- Clear visual feedback with a dropdown UI
 2025-01-20 12:35:43 -08:00
sabaimran
0c29c7a5bf Update layout and rendering of share page for hidden agent 2025-01-20 12:02:07 -08:00
sabaimran
000580cb8a Improve loading state when files not found and fix default state interpretation for model selector 2025-01-20 11:47:42 -08:00
sabaimran
235114b432 Fix agent data import across chat page + 2025-01-20 11:04:48 -08:00
sabaimran
d681a2080a Centralize use of useUserConfig and use that to retrieve default model and chat model options 2025-01-20 10:59:02 -08:00
sabaimran
a3fcd6f06e Fix import of AgentData from agentcard 2025-01-20 10:36:11 -08:00
sabaimran
d7800812ad Fix default states for the model selector 2025-01-20 10:18:09 -08:00
sabaimran
98baa93a31 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-chat-controls 2025-01-20 09:36:42 -08:00
sabaimran
696551b686 Add typing to operations in merge migration file 2025-01-20 08:36:40 -08:00
sabaimran
d1e7b5b234 Add a merge migration to resolve differences 2025-01-20 08:34:57 -08:00
sabaimran
1f59afe962 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-a-knowledge-base-page 2025-01-20 08:32:17 -08:00
sabaimran
016bcd7674 Revert page size to 10, rather than 1 2025-01-20 08:32:02 -08:00
sabaimran
8fe08eecce add --break-system-packages to bypass venv requirement 2025-01-20 00:21:27 -08:00
sabaimran
bf58d9430b downgrade postgres server pkg to 16 2025-01-20 00:15:56 -08:00
sabaimran
95ad1f936e upgrade postgres server to 17 2025-01-20 00:10:20 -08:00
sabaimran
a214bd4100 upgrade pg server dev version to 15 2025-01-20 00:05:35 -08:00
sabaimran
82ff74cfa9 Run on container with ubuntu latest for pytest gh action workflow 2025-01-19 23:57:57 -08:00
sabaimran
83d856f97d Add some basic pagination logic to the knowledge page to prevent overloading the Api or the client 2025-01-19 22:48:36 -08:00
sabaimran
59ee6e961a Only set hasmodified to true during model select if different from original model 2025-01-19 18:19:48 -08:00
sabaimran
e982398c2c Weird spacing issue resolve (it was because of the footer in the collapsed state still having some width) 2025-01-19 18:16:43 -08:00
sabaimran
dbce039033 Revert the fixed hack to hide horizontal spacing issue with sidebar because it breaks the animation on closed. Sigh. 2025-01-19 18:11:53 -08:00
sabaimran
0d38cc9753 Handle further edge cases when setting chat agent data and fix alignment of chat input / side panel 2025-01-19 17:59:37 -08:00
sabaimran
b248123135 Hook up hidden agent creation and update APIs to the UI 
- This allows users to initiate hidden agent creation from the side bar directly. Any updates can easily be applied to the conversation agent.
 2025-01-19 17:36:30 -08:00
sabaimran
be11f666e4 Initialize the concept in the backend of hidden agents 
A hidden agent basically allows each individual conversation to maintain custom settings, via an agent that's not exposed to the traditional functionalities allotted for manually created agents (e.g., browsing, maintenance in agents page).

This will be hooked up to the front-end such that any conversation that's initiated with the default agent can then be given custom settings, which in the background creates a hidden agent. This allows us to repurpose all of our existing agents infrastructure for chat-level customization.
 2025-01-19 12:16:37 -08:00
sabaimran
0a0f30c53b Update relevant agent tool descriptions 
Remove text (as by default, must output text), and improve the Notes description for clarity
 2025-01-19 12:14:28 -08:00
sabaimran
f10b072634 update looks & feel of chat side bar with model selector, checkboxes for tools, and actions (not yet implemented) 2025-01-19 12:13:37 -08:00
sabaimran
7837628bb3 Update existing agentData imports 2025-01-19 12:12:23 -08:00
sabaimran
7998a258b6 Add additional ui components for tooltip, checkbox 2025-01-19 12:08:02 -08:00
sabaimran
00370c70ed Consolidate the AgentData Type into the agentCard 2025-01-19 12:06:54 -08:00
Debanjum
fde71ded16 Upgrade web app dependencies 2025-01-19 13:44:59 +07:00
Debanjum
2d4633d298 Use encoded email, otp in login URL in email & web app sign-in flow 
Previously emails with url special characters would not get
successfully identified for login. Account creation was fine due to
email being in POST request body. But login with such emails did not
work due to query params not being escaped before being sent to server

This change escapes both the code and email in login URL sent to
server. So login with emails containing special characters like
`email+khoj@gmail.com' works. It fixes both the URL web app sent by
web app directly and the magic link sent to users to their email

This change also fixes accessibility issue of having a DialogTitle in
DialogContent for screen readers.

Resolves #1090
 2025-01-19 13:11:23 +07:00
Debanjum
51f3af11b5 Fix Qwen 2.5 14B model source to use Q4_K_M quantized model 
The official Qwen2.5 14B model doesn't mention standard quantization
suffixes like Q4_K_M, so doesn't work with Khoj
 2025-01-19 12:27:35 +07:00
sabaimran
f99bd3f1bc Merge branch 'master' of github.com:khoj-ai/khoj into features/add-chat-controls 2025-01-17 17:49:08 -08:00
sabaimran
af9e906cb5 Use python3 instead of python when running pip install commands in gh actions 2025-01-17 17:48:42 -08:00
sabaimran
c80e0883ee Use python3 instead of python for install pip commands in GH action 2025-01-17 17:36:46 -08:00
sabaimran
b8c866014d Improve instruction description for the agent command description for notes 2025-01-17 17:19:59 -08:00
sabaimran
7481f78f22 Remove unused API request 2025-01-17 17:18:47 -08:00
sabaimran
5aadba20a6 Add backend support for hidden agents (not yet enabled) 2025-01-17 16:46:37 -08:00
sabaimran
2fa212061d Add a ride hand side bar for chat controls 2025-01-17 16:45:50 -08:00
Tuğhan Belbek
849348e638 Handle additional HTTP redirect status code 308 in scheduled chat requests (#1088) 
Closes #1067
 2025-01-16 07:03:43 -08:00
Debanjum
00843f4f24 Release Khoj version 1.34.0 2025-01-16 12:11:28 +07:00
Debanjum
ad27f34c96 Support online search using Google Search API (#1087) 
Add official Google Search API as an online search provider.
We currently support Serper.dev and Searxng as online search providers.
 2025-01-16 11:41:59 +07:00
Debanjum
a649b03658 Support online search using Google Search API 2025-01-16 11:39:03 +07:00
Sam Ho
f8f159efac feat: add turnId handling to chat messages and history 2025-01-16 00:44:16 +00:00
sabaimran
42d4d15346 Merge pull request #1054 from khoj-ai/features/add-support-for-mermaidjs 
We've been having issues generating diagrams with Excalidraw that are any degree of complexity. By contrast, LLMs are able to handle Mermaid.js syntax a lot better, as it's much more forgiving and has a simpler declarative style. Refer to https://mermaid.js.org/.

Update so that new diagrams are generated with Mermaid.js, while old diagrams generated with Excalidraw can still be viewed.
 2025-01-15 11:55:12 -08:00
Debanjum
e2b2b3415e Fix handling of inline base64 images by Obsidian, Desktop clients 
Fix for #1082 pushed down adding the `data:image/webp;base64' prefix
of the base64 images to the server image gen API. But the code on the
Obsidian and Desktop client still add these prefixes.

This change stops the Desktop, Obsidian clients from adding the prefix
as it is being handled by the API now. It should resolve showing
images inline in those clients as well
 2025-01-15 23:34:23 +07:00
Debanjum
2e585efd2f Fix end with newline styling issue in style.css to pass lint checks 2025-01-15 19:43:02 +07:00
Debanjum
ed18c04576 Fix wrapping base64 generated image for inline display 
Resolves #1082
 2025-01-15 19:19:31 +07:00
Debanjum
f8b887cabd Allow using OpenAI (compatible) API for Speech to Text transcription 2025-01-15 19:19:31 +07:00
Debanjum
182c49b41c Prefer explicitly configured OpenAI API url, key for image gen model 
Previously we'd use the general openai client, even if the image
generation model has a different api key and base url set.

This change uses the openai config of the image generation models when
set. Otherwise it fallbacks to use the first openai api provider set
 2025-01-15 19:19:31 +07:00
Debanjum
24204873c8 Use same openai base url env var name as the official openai client 
This eases re-use of the OpenAI API across all openai clients,
including chat, image generation, speech to text.

Resolves #1085
 2025-01-15 19:19:30 +07:00
Debanjum
63dd3985b5 Support using Embeddings Model exposed via OpenAI (compatible) API (#1051) 
This change adds the ability to use OpenAI, Azure OpenAI or any embedding model exposed behind an OpenAI compatible API (like Ollama, LiteLLM, vLLM etc.).

Khoj previously only supported HuggingFace embedding models running locally on device or via HuggingFaceW inference API endpoint. This allows using commercial embedding models to index your content with Khoj.
 2025-01-15 17:39:54 +07:00
Debanjum
a6bf6803b6 Add docs on how to add, edit search model configs when self-hosting 2025-01-15 17:30:18 +07:00
Debanjum
92a1ec7afc Do not auto restart khoj docker services by default 
Let folks who want to add that, add it manually if they want to. It
creates too much noise for folks having trouble with self-host setup
 2025-01-15 13:09:50 +07:00
Debanjum
85c537a1de Set default PORT arg in Dockerfile to default Khoj port, 42110 2025-01-15 13:09:50 +07:00
Debanjum
9355381fac Catch error in call to data sources, output format selection tool AI 
Previously if the call to this tool AI failed, the API call would
non-gracefully fail on server. This would leave the client hanging in
a wierd state (e.g with spinner running on web app with no indication
of issue).

Also do not show filters in query debug log lines when no filters in query
 2025-01-15 13:09:50 +07:00
Debanjum
24ab8450ba Handle scenario where read chat stream error is not json on web app 2025-01-15 13:09:50 +07:00
sabaimran
0b775c77d3 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-a-knowledge-base-page 2025-01-13 15:07:59 -08:00
Sam Ho
fc6fab4cce chore: fix format issue from pre-commit hook - trailing-whitespace and end-of-file-fixer 2025-01-13 20:07:48 +00:00
sabaimran
7f329e7e9d Fix configuration of name field for chatmodel options during initalization 2025-01-12 22:37:08 -08:00
sabaimran
1a00540ee9 Improve error handling in mermaid chart rendering 2025-01-12 22:36:31 -08:00
Osama Ata
96e3d0a7b9 Fix stale lmstudio documentation to set ai model api via admin panel (#1075) 
Use new name `Ai Model API` instead of  `OpenAI Processor Conversation Config`
 2025-01-12 03:06:01 -08:00
Yash-1511
27165b3f4a fix: review suggestions 2025-01-12 15:12:14 +05:30
Debanjum
6bd9f6bb61 Give a shorter, simpler name to github workflow to deploy docs 2025-01-12 10:54:56 +07:00
Sam Ho
93687f141a feat: do not show delete button on system messages 2025-01-11 17:35:57 +00:00
Sam Ho
a9c180d85f feat: add delete chat message action to the Obsidian plugin 2025-01-11 17:19:40 +00:00
Debanjum
51a774c993 Add contrast to setting card inputs in dark mode on web app 2025-01-11 14:50:47 +07:00
Debanjum
9e8b8dc5a2 Toggle showing api key on web settings page via a visibility toggle 
- Background
  Access to the clipboard API is disabled by certain browsers in non
  localhost http scenarios for security reasons.

  So the copy API key button doesn't work when khoj is self-hosted
  with authentication enabled at a non localhost domain

- Change
  This change enables copying API keys by manual text highlight + copy
  if copy button is disabled

Resolves #1070
 2025-01-11 14:50:47 +07:00
Debanjum
25c39bd7da Extract api keys setting card into separate component on web app 2025-01-11 14:50:46 +07:00
sabaimran
c30047e859 Fix Obsidian style.css 2025-01-10 22:18:44 -08:00
sabaimran
da2b89e46a Merge branch 'master' of github.com:khoj-ai/khoj into features/add-a-knowledge-base-page 2025-01-10 22:18:14 -08:00
sabaimran
f170487338 Fix apostrophe in the add documents modal 2025-01-10 21:58:17 -08:00
sabaimran
be4b091a21 Add new line to styles.css 2025-01-10 21:52:52 -08:00
sabaimran
f398e1eb0c Add codeblock rendering for the mermaidjs diagram in obsidian 2025-01-10 21:46:39 -08:00
Debanjum
6e955e158b Use normalized email address for new users 
Not check email deliverability for now to allow air-gapped usage or
authenticated/multi-user setups with admin managed otp

Closes #1069
 2025-01-11 12:28:40 +07:00
sabaimran
c441663394 Merge branch 'master' of github.com:khoj-ai/khoj into features/add-support-for-mermaidjs 2025-01-10 21:25:33 -08:00
sabaimran
85c34a5f0f Merge pull request #1018 from hjamet/master 
This PR delivers comprehensive improvements to the Khoj plugin across multiple key areas:

🔍 Search Enhancements:
- Added visual loading indicators during search operations
- Implemented color-coded results to distinguish between vault and external files
- Added abort logic for previous requests to improve performance
- Enhanced search feedback with clear status indicators
- Improved empty state handling

🔄 Synchronization Improvements:
- Added configurable sync interval setting in minutes
- Implemented manual "Sync new changes" command
- Enhanced sync timer management with automatic restart
- Improved notification system for sync operations

📁 Folder Management:
- Added granular folder selection for sync
- Implemented intuitive folder suggestion modal
- Enhanced folder list visualization

💅 UI/UX Improvements:
- Added loading animations and spinners
- Enhanced search results visualization with color coding
- Refined chat interface styling
- Improved overall settings panel organization

🔧 Technical Improvements:
- Refactored search and synchronization logic
- Implemented proper request cancellation
- Enhanced error handling and user feedback
- Improved code organization and maintainability
 2025-01-10 21:24:12 -08:00
sabaimran
57545c1485 Fix the migration script to delete orphaned fileobjects 
- Remove knowledge page from the sidebar
- Improve speed and rendering of the documents in the search page
 2025-01-10 21:06:48 -08:00
sabaimran
d77984f9d1 Remove separate knowledge base file - consolidated in the search page 2025-01-10 18:57:38 -08:00
sabaimran
f2c6ce2435 Improve rendering of the file objects and sort files by updated_date 2025-01-10 18:18:15 -08:00
Patrick Jackson
6e0c767ff0 Use the configured OpenAI Base URL for Automations (#1065) 
This change makes Automations (and possibly other entrypoints) use the configured OpenAI-compatible server if that has been set. Without this change it tries to use the hardcoded OpenAI provider.
 2025-01-10 17:17:51 -08:00
sabaimran
454a752071 Initial commit: add a dedicated page for managing the knowledge base 
- One current issue in the Khoj application is that managing the files being referenced as the user's knowledge base is slightly opaque and difficult to access
- Add a migration for associating the fileobjects directly with the Entry objects, making it easier to get data via foreign key
- Add the new page that shows all indexed files in the search view, also allowing you to upload new docs directly from that page
- Support new APIs for getting / deleting files
 2025-01-10 16:24:50 -08:00
Debanjum
1b5826d8b6 Support using Embeddings Model exposed via OpenAI (compatible) API 2025-01-10 23:48:04 +07:00
Debanjum
65f1c27963 Remove old, big warning about Khoj not configured on server init 
- Just say using default config. This old khoj.yml settings mechamism
  isn't standard, so not having a configured khoj.yml isn't a concern
- Deep link to desktop download instead of the whole download page as
  android etc. are also on it, which don't help with syncing docs
 2025-01-10 23:46:27 +07:00
Debanjum
3cc6597b49 Support Azure OpenAI API endpoint (#1048) 
OpenAI chat models deployed on Azure are (ironically) not OpenAI API compatible endpoints.
This change enables using OpenAI chat models deployed on Azure with Khoj.
 2025-01-10 08:35:03 -08:00
sabaimran
bac90ad69d Upgrade deploy-pages action to vv4 2025-01-09 19:04:31 -08:00
Debanjum
2069f571c8 Upgrade upload-artifact gh action to v4 as <=v3 deprecated 
This started failing github workflow jobs
 2025-01-10 00:41:24 +07:00
Debanjum
dd63bd8bcf Fix dark mode dropdown colors of phone no. country code on web settings page 
Resolves #1046
 2025-01-10 00:10:51 +07:00
Debanjum
bb6a6cbe19 Restart Khoj docker services unless stopped. Remove default network 
- Seems less aggressive to use unless-stopped versus always.
- Default network is used anyway, so doesn't seem necessary to specify
 2025-01-09 22:08:48 +07:00
Debanjum
01d27f5220 Do not show user logout button on web app side pane in anoymous mode 
Refer
https://github.com/khoj-ai/khoj/issues/1050#issuecomment-2579119234
 2025-01-09 21:18:50 +07:00
Debanjum
a739936563 Mark Github integration as unmaintained in documentation 
Also mention what the Reflective Questions table is about

Resolves #1059
 2025-01-09 20:42:38 +07:00
Debanjum
1eaa54b0ae Make PAT token requirement optional for Github indexing for now 
The github integration has not been tested and may still be broken.
This change at least makes it easier to add repositories without
needing a PAT token if/when it does work.
 2025-01-09 20:42:38 +07:00
sabaimran
ec02757fd1 Add an export feature along with the mermaid diagram. Add sidebar to loading page. 2025-01-08 23:53:58 -08:00
sabaimran
889f34c7bf Adjust typing and error handling for incorrect diagrams 2025-01-08 23:22:16 -08:00
sabaimran
11fcf2f299 Remove dangling response type 2025-01-08 22:10:59 -08:00
sabaimran
6b0a49b12d Add the mermaid package and apply front-end parsing 
- Add the mermaid package and apply front-end parsing for interpreting the diagrams. Retain processing of the excalidraw type for backwards compatibility
 2025-01-08 22:09:35 -08:00
sabaimran
539ce99343 Add backend support for parsing and processing and storing mermaidjs diagrams 
- Replace default diagram output from excalidraw to mermaid
- Retain typing of the excalidraw type for backwards compatibility in the chatlog
 2025-01-08 22:08:40 -08:00
sabaimran
c448c49811 Clean-up some code on homepage and disable initial card animations because of jitter 2025-01-08 22:07:23 -08:00
Ikko Eltociear Ashimine
9ce9f02886 Fix typo in Admin Doc (#1034) 
appropiate -> appropriate
 2025-01-08 21:27:34 -08:00
sabaimran
ad5f0c7a02 Merge pull request #1029 from DPS0340/master 
Improve docker-compose.yml

- Do not expose dependencies on host internet
- Put all services on the same network
 2025-01-08 10:42:47 -08:00
Henri Jamet
f42b0cb08c Refactor comments and CSS for improved clarity in Khoj plugin 
- Translated comments from French to English for better accessibility and understanding.
- Updated CSS comment for loading animation to reflect the change in language.
- Enhanced code readability by ensuring consistent language usage across multiple files.
- Improved user experience by clarifying the purpose of various functions and settings in the codebase.
 2025-01-08 09:31:43 +01:00
sabaimran
875cdde9b9 Release Khoj version 1.33.2 2025-01-07 15:32:18 -08:00
sabaimran
5c5c4a6bbc Add help text for Enterprises in the README 2025-01-07 14:55:57 -08:00
sabaimran
8d028e10c6 Fix populating login url in sign in email 2025-01-07 14:53:00 -08:00
omahs
36bdaedd2d Fix typos in Khoj Docs (#1033) 2025-01-07 15:55:57 +07:00
sabaimran
25c1c1c591 Release Khoj version 1.33.1 2025-01-06 09:08:01 -08:00
sabaimran
689d9d8b3a Update formatting in admin.py and utils.py 2025-01-06 09:07:28 -08:00
thinker007
aa442c28eb Handle reporting chat estimated cost when some fields unavailable (#1026) 
Fix AttributeError: 'NoneType' object has no attribute 'model_extra'

* cost = chunk.usage.model_extra.get("estimated_cost", 0) if hasattr(chunk, "usage") and chunk.usage else 0  # Estimated costs returned by DeepInfra API
 2025-01-06 09:03:49 -08:00
sabaimran
4aed6f7e08 Add a link around the header khojlogotype to go home 2025-01-06 08:55:00 -08:00
Debanjum
266d274e21 Make automation should_notify check robust to non json mode chat models 
Use clean_json to handle automation should_notify check For gemini and
other chat models where enforcing json mode is problematic, not supported
 2025-01-06 20:16:32 +07:00
Debanjum
9a5e3583cf Remove bullet styling only from sidebar items on web app 
Previous fix had removed bullet styling from all components in web
app. This made chat messages on the web app lose bullet styling too.
 2025-01-06 20:15:42 +07:00
Debanjum
dc0bc5bcca Evaluate information retrieval quality using eval script 
- Encode article urls in filename indexed in Khoj KB
  Makes it easier for humans to compare, trace retrieval performance
  by looking at logs than using content hash (which was previously
  explored)
 2025-01-06 13:19:52 +07:00
Debanjum
daeba66c0d Optionally pass references used by agent for response to eval scorers 
This will allow the eval framework to evaluate retrieval quality too
 2025-01-06 13:19:52 +07:00
Debanjum
8231f4bb6e Return accuracy as decision to generalize across IR & standard scorers 2025-01-06 13:19:52 +07:00
Jiho Lee
c1c086e431 fix: Use localhost on SEARXNG_BASE_URL 
Co-authored-by: sabaimran <65192171+sabaimran@users.noreply.github.com>
 2025-01-06 13:32:06 +09:00
sabaimran
eb9aadf72a Add an Obsidian README documentation for development 2025-01-05 19:06:27 -08:00
sabaimran
e89e49818b Merge pull request #1028 from ReallyVirtual/patch-1 
Update image_generation.md
 2025-01-05 13:53:00 -08:00
sabaimran
616cc189d1 Remove bullet points from li styling explicitly 2025-01-05 13:52:08 -08:00
sabaimran
a5705a5aa6 After agent prompt safe check is parsed as json, load it into a json object 2025-01-05 13:40:20 -08:00
Yash-1511
f306159a5a feat: add autocomplete suggestions feature in search page 2025-01-05 17:30:00 +05:30
Jiho Lee
a5c7315874 feat: Improve docker-compose.yml 
- Remove host port mappings on dependencies
- Add 'restart: always'
- Add default network for lookup by docker dns
 2025-01-05 15:17:50 +09:00
Sohaib Athar
95a2387e9b Update image_generation.md 
Fixed typo
 2025-01-05 07:25:41 +05:00
sabaimran
756f4a2a66 Update regeneration logic to run for all entries now that we have a single search model ID 2025-01-03 15:16:10 -08:00
Debanjum
33f85b4a55 Fix title of Query Filters documentation 2025-01-01 18:56:16 +08:00
sabaimran
0ef787b57c Set waitBeforeSeconds to 0 2024-12-30 15:08:34 -08:00
Debanjum
90b4e03454 Pull out query filters as top level documentation page 
- Note perf eval from 2022
- Update links to query-filters in docs
- Fix links
- Update image model docs
 2024-12-30 14:35:05 -08:00
sabaimran
8f69eb949b Release Khoj version 1.33.0 2024-12-29 10:39:26 -08:00
Henri Jamet
1aff78a969 Enhance Khoj plugin search functionality and loading indicators 
- Added visual loading indicators to the search modal for improved user experience during search operations.
- Implemented logic to check if search results correspond to files in the vault, with color-coded results for better clarity.
- Refactored the getSuggestions method to handle loading states and abort previous requests if necessary.
- Updated CSS styles to support new loading animations and result file status indicators.
- Improved the renderSuggestion method to display file status and provide feedback for files not in the vault.
 2024-12-29 16:19:42 +01:00
Henri Jamet
7d28b46ca7 Implement sync interval setting and enhance synchronization timer in Khoj plugin 
- Added a new setting for users to configure the sync interval in minutes, allowing for more flexible automatic synchronization.
- Introduced methods to start and restart the synchronization timer based on the configured interval.
- Updated the synchronization logic to use the user-defined interval instead of a fixed 60 minutes.
- Improved code readability and organization by refactoring the sync timer logic.
 2024-12-29 13:23:08 +01:00
Henri Jamet
c5c9e0072c Enhance Khoj plugin settings and UI for folder synchronization 
- Added a new setting to manage sync folders, allowing users to specify which folders to sync or to sync the entire vault.
- Implemented a modal for folder suggestions to facilitate folder selection.
- Updated the folder list display to show currently selected folders with options to remove them.
- Improved CSS styles for chat interface and folder list for better user experience.
- Refactored code for consistency and readability across multiple files.
 2024-12-29 13:07:21 +01:00
Henri Jamet
833ab52986 Add manual sync command to Khoj plugin 
- Introduced a new command 'Sync new changes' to allow users to manually synchronize new modifications.
- The command updates the content index without regenerating it, ensuring only new changes are synced.
- User-triggered notifications are displayed upon successful sync.
 2024-12-29 12:51:52 +01:00
Debanjum
1a43ca75f3 Update to latest jinja python package dependency 2024-12-27 01:44:41 -08:00
Debanjum
ca197ba9ba Improve Automation Flexibility and Automation Email Format (#1016) 
- Format AI response to send in automation email
- Let Khoj infer chat query based on user automation query
- Decide if automation emails should be sent based on response
  - Fix the `to_notify_or_not` decider AI
  - Ask reason before decision to improve to_notify decider AI
- Show error message on web app when fail to create/update automation
 2024-12-27 01:36:38 -08:00
Debanjum
90685ccbb0 Show error message if update automation via web app fails 2024-12-26 22:21:56 -08:00
Debanjum
c4bb92076e Convert async create automation api endpoints to sync 2024-12-26 21:59:55 -08:00
Debanjum
9674de400c Format AI response to send in automation email 
Previously we sent the AI response directly. This change post
processes the AI response with the awareness that it is to be sent to
the user as an email to improve rendering and quality of the emails.
 2024-12-26 21:04:50 -08:00
Debanjum
6d219dcc1d Switch to let Khoj infer chat query based on user automation query 
This tries to decouple the automation query from the chat query. So
the chat model doesn't have to know it is running in an automation
context or figure how to notify user or send automation response. It
just has to respond to the AI generated `query_to_run' corresponding
to the `scheduling_request` automation by the user.

For example, a `scheduling_request' of `notify me when X happens'
results in the automation calling the chat api with a `query_to_run`
like `tell me about X` and deciding if to notify based on information
gathered about X from the scheduled run. If these two are not
decoupled, the chat model may respond with how it can notify about X
instead of just asking about it.

Swap query_to_run with scheduling_request on the automation web page
 2024-12-26 21:04:50 -08:00
Debanjum
3600a9a4f3 Ask reason before decision to improve to_notify decider automation AI 
Previously it just gave a decision. This was hard to debug during
prompt tuning. Asking for reason before decision improves models
decision quality.
 2024-12-26 21:04:50 -08:00
Debanjum
dcc5073d16 Fix the to_notify decider automation chat actor. Add detailed logging 2024-12-26 21:04:50 -08:00
sabaimran
03b4667acb Merge pull request #1017 from khoj-ai/features/update-home-page 
- Rather than chunky generic cards, make the suggested actions more action oriented, around the problem a user might want to solve. Give them follow-up options. Design still in progress.
 2024-12-24 12:12:37 -08:00
sabaimran
5985ef4c7c Further improve prompts 2024-12-24 12:11:27 -08:00
sabaimran
012e0ef882 Add tooltip for file input ref. 2024-12-24 11:20:39 -08:00
sabaimran
a58b3b4a37 Remove some of the step one suggestions 2024-12-24 10:52:42 -08:00
sabaimran
cf78f426d3 Merge branch 'master' of github.com:khoj-ai/khoj into features/update-home-page 2024-12-24 09:52:06 -08:00
sabaimran
d27d291584 Merge pull request #1015 from khoj-ai/features/clean-up-authenticated-data 
- De facto, was being assumed everywhere if authenticatedData is null, that it's not logged in. This isn't true because the data can still be loading. Update the hook to send additional states.
- Bonus: Delete model picker code and a slew of unused imports.
 2024-12-24 09:51:39 -08:00
sabaimran
cd4cf4f9f6 Merge pull request #1014 from khoj-ai/features/improve-agent-management 
- Add support for seeing all steps of the agent modification flow via tabs at the top of the modal
- Separate knowledge base & tool selection into two separate parts
 2024-12-24 09:50:38 -08:00
sabaimran
3e6ba45cbe Merge pull request #1013 from khoj-ai/features/use-sidebar 
Use the [shadcn sidebar](https://ui.shadcn.com/docs/components/sidebar#sidebarmenusub) across Khoj and standardize how the side panel experience works across the app. This helps us generalize the code better, while re-using the same components. 

Deprecates current usage of the chat history side panel, replacing it with the new `appSidebar.tsx` component.

We'll eventually move out the `Manage Files` section and move it into a separate panel dedicated to chat-level controls.
 2024-12-24 09:50:09 -08:00
sabaimran
95fdbe13ae move all conversations button to bottom of side panel 2024-12-23 23:13:19 -08:00
sabaimran
d0256f267e Fix submit state for form with buttons sticky to button 2024-12-23 23:10:19 -08:00
Debanjum
0d5fc70aa3 Fix colors, title on create agent card 2024-12-23 20:01:56 -08:00
Debanjum
0b91383deb Make post oauth next url redirect more robust, handle q params better 2024-12-23 17:50:45 -08:00
Debanjum
17f8ba732d Autofocus on email input box when email sign-in selected on web app 2024-12-23 17:49:41 -08:00
sabaimran
fb111a944b Use disable_https flag instead of is_in_debug_mode to decide whether to redirect google auth request 2024-12-23 17:23:47 -08:00
sabaimran
3fd0c202ea Allow better spacing in the agent card and make the buttons sticky 2024-12-23 16:43:50 -08:00
sabaimran
c83709fdd1 Further clean up in home page initial cards experience 2024-12-22 18:11:19 -08:00
sabaimran
4c4f4401b1 Make suggestion cards a little more minimal 2024-12-22 11:54:27 -08:00
sabaimran
90b02b4cfe Merge branch 'features/clean-up-authenticated-data' of github.com:khoj-ai/khoj into features/update-home-page 2024-12-22 11:26:29 -08:00
sabaimran
798837378f Improve mobile friendliness and highlight missing necessary data 2024-12-22 11:02:50 -08:00
sabaimran
7032ccf130 Show create agent button when not logged in agents page 2024-12-22 10:01:15 -08:00
sabaimran
0fefbac89f Improve sidebar experience for not logged in state 2024-12-22 09:58:21 -08:00
sabaimran
60f80548a4 Remove unused span text 2024-12-22 09:21:40 -08:00
sabaimran
9f84f5dcc7 Give more breathing space to the sidebar footer 2024-12-22 09:18:05 -08:00
sabaimran
dc17272f70 Fix some spacing in the nav menu 2024-12-22 09:01:09 -08:00
sabaimran
45da6ec750 Separate the shorthand of each suggestion card from the prefilled text 2024-12-21 20:46:57 -08:00
sabaimran
bf405f50d7 Fix spacing of main content in the settings page 2024-12-21 20:46:21 -08:00
sabaimran
62dd4c55d4 Further improve UX of the suggestion cards 2024-12-21 20:10:54 -08:00
sabaimran
8c9c57e060 Merge branch 'features/clean-up-authenticated-data' of github.com:khoj-ai/khoj into features/update-home-page 2024-12-21 19:32:07 -08:00
sabaimran
8c6b4217ae Set width of chat layout to 100% 2024-12-21 19:29:37 -08:00
sabaimran
a17cc9db38 Fix handling 403 forbidden error from auth response 2024-12-21 19:19:35 -08:00
sabaimran
95826393e1 Update the home page suggestion cards 
- Rather than chunky generic cards, make the suggested actions more action oriented, around the problem a user might want to solve. Give them follow-up options. Design still in progress.
 2024-12-21 18:57:19 -08:00
Debanjum
8d129c4675 Bump default max prompt size for commercial chat models 2024-12-21 17:31:05 -08:00
Debanjum
37ae48d9cf Add support for OpenAI o1 model 
Needed to handle the current API limitations of the o1 model.
Specifically its inability to stream responses
 2024-12-21 17:25:32 -08:00
sabaimran
2c7c16d93e Fix conditional reference to is mobile width hook 2024-12-21 12:48:29 -08:00
sabaimran
e9dae4240e Clean up all references to authenticatedData 
- De facto, was being assumed everywhere if authenticatedData is null, that it's not logged in. This isn't true because the data can still be loading. Update the hook to send additional states.
- Bonus: Delete model picker code and a slew of unused imports.
 2024-12-21 08:45:43 -08:00
sabaimran
b1c5c5bcc9 Fix path to component library in shadcn sidebar 2024-12-20 15:48:57 -08:00
sabaimran
cc7fd1163f Use tabs to separate sections in the agent mod form 
- Add knowledge base as a separate section, apart from tools
- This makes it easier to navigate the different components quickly
 2024-12-20 14:41:15 -08:00
sabaimran
078753df30 Add chatwoot to the frame-src CSP 2024-12-20 13:33:48 -08:00
sabaimran
efe812e323 Add componets for tabs in agent page 2024-12-20 13:33:20 -08:00
sabaimran
ba792c02ba Improve share chat UI for alignment 2024-12-20 12:28:31 -08:00
sabaimran
7770caa793 Add side bar inset to home page. Simplify automations card. 2024-12-20 11:37:23 -08:00
sabaimran
b8a9dcd600 Improve mobile layout with sidebar inset and fix dark mode logo 2024-12-19 23:23:52 -08:00
sabaimran
b1880d9c9d Add side bar to search page 2024-12-19 22:46:50 -08:00
sabaimran
02c503e966 Further improve mobile layout with side panel 2024-12-19 22:00:32 -08:00
sabaimran
43331f7730 Remove unused css classes 2024-12-19 21:36:35 -08:00
sabaimran
b644bb8628 Further improve mobile friendliness 2024-12-19 21:34:04 -08:00
sabaimran
9d5480d886 Improve mobile friendlinses across chat and home pages. 2024-12-19 20:33:53 -08:00
sabaimran
68af10c805 Use the new shadcn sidebar for khoj nav and actions 
- Use the sidebar across all pages to quickly navigate through the app, access settings, and go to past chats
- Pending: mobile friendliness
 2024-12-19 20:10:03 -08:00
sabaimran
7eb15bf0a9 Update shadcn components 2024-12-19 14:33:36 -08:00
sabaimran
a4aeb9fdf3 Simplify the home page color scheme and overall design 2024-12-19 14:02:53 -08:00
Debanjum
0ae21e5628 Use icons, not text labels, for sidebar nav items on docs website 2024-12-17 20:49:09 -08:00
sabaimran
cafe1b0655 Remove error log line for payload inclusion 2024-12-17 19:48:22 -08:00
Debanjum
1e3f452d15 Handle sharing old conversation publically even if they have no slug 
New conversation have a slug, but older conversation may not. This
change allows those older conversations to still be shareable by using
a random uuid for constructing their url instead
 2024-12-17 19:44:36 -08:00
Debanjum
90b7ba51a4 Make agent prompt safety checker more forgiving and concise 
- Some of the instructions were duplicated (e.g illegal, harmful)
- Return format requested was inconsistent
- Safety prompt felt overtly prudish which lowered their utility.
  Make it laxer for now, add checks later if required
 2024-12-17 19:44:36 -08:00
sabaimran
bcee2ea01a Release Khoj version 1.32.2 2024-12-17 16:03:29 -08:00
sabaimran
92144c8102 Remove release step in todesktop flow, since we need to run releases manually now 
- Leaving it commented out for the time being so we can revisit automating this later
 2024-12-17 16:02:45 -08:00
sabaimran
f291884921 If not in debug mode, force google auth to use the https protocol 2024-12-17 15:44:18 -08:00
sabaimran
bcc1bc6854 Log the payload sent temporarily in order to help with debugging 2024-12-17 15:37:42 -08:00
sabaimran
7ca2553d17 Update login popup copy 2024-12-17 15:20:44 -08:00
sabaimran
ef99d8c28e Add more descriptive error logs when google auth token verification fails 2024-12-17 15:18:38 -08:00
Debanjum
60d55a83c4 Use khoj logo via url in readme to load in other locations like pypi 2024-12-17 14:00:53 -08:00
Debanjum
10bd56d2b9 Attest Khoj pypi package by upgrading pypi publish gh action 
- Print hash in CI to ease verifying ci built python package matches
  khoj package published on pypi
- Newer pypi publish github action should speed up workflow by ~30s
 2024-12-17 13:40:39 -08:00
sabaimran
2e80a1ce7c Release Khoj version 1.32.1 2024-12-17 13:28:00 -08:00
Debanjum
df15f00243 Tag docker images with latest tag in dockerize workflow on release 2024-12-17 13:18:51 -08:00
sabaimran
ded168dae3 Release Khoj version 1.32.0 2024-12-17 12:29:20 -08:00
sabaimran
f6abfcfa6b Use latest release version for pypi gh action to publish 2024-12-17 12:19:42 -08:00
Debanjum
c20364efcb Upgrade web app next.js, shadcn and other package dependencies 2024-12-17 11:15:22 -08:00
Debanjum
63d2c6f35a Allow research mode and other conversation commands in automations (#1011) 
Major
---
Previously we couldn't enable research mode or use other slash
commands in automated tasks.

This change separates determining if a chat query is triggered via
automated task from the (other) conversation commands to run the query
with.

This unlocks the ability to enable research mode in automations
apart from other variations like /image or /diagram etc.

Minor
---
- Clean the code to get data sources and output formats
- Have some examples shown on automations page run in research mode now
 2024-12-17 00:44:51 -08:00
sabaimran
3b050a33bb Include resend as a default dependency, rather than restricting to prod 2024-12-16 22:24:41 -08:00
sabaimran
741e9f56f9 Update admin button for getting login url to include user email 2024-12-16 22:16:05 -08:00
Debanjum
88aa8c377a Support online search with Searxng as zero config, self-hostable solution (#1010) 
This allows online search to work out of the box again 
for self-hosting users, as no auth/api key setup required.

Docker users do not need to change anything in their setup flow.
Direct installers can setup Searxng locally or use public instances if
they do not want to use any of the other providers (like Jina, Serper)

Resolves #749. Resolves #990
 2024-12-16 18:59:09 -08:00
sabaimran
7677bb14f8 Update docs home page 2024-12-16 18:48:42 -08:00
sabaimran
1bd0d46b3d Update the theme color used in our docs to match the theme in our emails 2024-12-16 18:42:50 -08:00
sabaimran
432e901087 Update the online search self-hosting instructions to reflect new setup 2024-12-16 18:37:20 -08:00
sabaimran
af2553a890 Remove the Searxng API key env variable for simplicity 2024-12-16 18:36:55 -08:00
sabaimran
19d80d190e Add latest tag to the khoj cloud description for prod 2024-12-16 17:56:12 -08:00
sabaimran
d62cc0d539 Merge branch 'master' of github.com:khoj-ai/khoj into support-online-search-via-searxng 2024-12-16 17:55:06 -08:00
sabaimran
6f3218f487 Merge pull request #1008 from khoj-ai/features/new-sign-in-page 
- Make it easier to quickly create the account without losing track of where you are
- Show some capabilities before you sign on
 2024-12-16 17:54:43 -08:00
sabaimran
753859fbe0 Make the docs and github buttons on the sign in email less prominent 2024-12-16 17:49:13 -08:00
sabaimran
20888d3930 Clarify some of the language in the sign in email 2024-12-16 17:47:12 -08:00
sabaimran
13e7455f56 Use lowercase c in click 2024-12-16 17:44:08 -08:00
sabaimran
d17a9ba4c8 Fix return data for expired code user 2024-12-16 17:33:53 -08:00
sabaimran
efb0b9f495 Gracefully handle error when user login code is expired 2024-12-16 16:47:54 -08:00
sabaimran
064f7e48ca Update various copy texts for OG metadata and such 2024-12-16 16:40:46 -08:00
sabaimran
28b8f9105d Remove parenthetical from email template 2024-12-16 13:40:41 -08:00
Debanjum
9d02978f6e Support online search with Searxng as turnkey, self-hostable solution 
This allows online search to work out of the box again for
self-hosting users, as no auth/api key setup required.

Docker users do not need to change anything in their setup flow.
Direct installers can setup searxng locally or use public instances if
they do not want to use any of the other providers (like Jina, Serper)

Resolves #749. Resolves #990
 2024-12-16 12:53:38 -08:00
Debanjum
9c64275dec Auto redirect requests to use HTTPS if server is using SSL certs 2024-12-16 12:53:38 -08:00
sabaimran
ae9750e58e Add rate limiting to OTP login attempts and update email template for conciseness 2024-12-16 11:59:45 -08:00
sabaimran
b7783357fa Decrease timeout limit for verification codes to 5 minutes 2024-12-16 09:07:34 -08:00
sabaimran
5d3da3340f Include email in verification API 2024-12-15 13:54:41 -08:00
sabaimran
8e3313156e Simplify the magic link email a little bit 2024-12-14 11:19:31 -08:00
sabaimran
6a56140360 Allow users to directly enter their unique code when logging in 
- Code automatically becomes invalid after 30 minutes
 2024-12-14 11:06:05 -08:00
sabaimran
c25174e8d4 Apply more finished styling to login features, make the pop-up mobile friendly 2024-12-14 09:46:19 -08:00
sabaimran
73c1fe6ae1 Add text overlay to caption the different assets 2024-12-13 13:45:31 -08:00
Debanjum
132f2c987a Make Khoj email sender configurable for all email variants 
The welcome, feedback and automation emails were still using the Khoj
domain, which wouldnt work for self-hosted users with their RESEND key

Resolves #1004
 2024-12-13 12:25:23 -08:00
sabaimran
f1fb4525c6 Remove old images 2024-12-13 11:31:14 -08:00
sabaimran
d5681ad1a2 Update image assets to sign up prompt 2024-12-13 11:30:14 -08:00
sabaimran
62545a9af3 Update package path for pypi ci export 2024-12-12 21:25:21 -08:00
sabaimran
e74e922cea Update file path of python installation 2024-12-12 16:50:32 -08:00
sabaimran
144f283a25 Maintain old login page for posterity and associated API 2024-12-12 16:23:44 -08:00
sabaimran
4697daeb1a Improve opengraph metadata across front-end pages 2024-12-12 15:56:43 -08:00
sabaimran
dfc150c442 Merge branch 'master' of github.com:khoj-ai/khoj into features/new-sign-in-page 2024-12-12 15:43:06 -08:00
sabaimran
ad3f8a33d1 Add a static login footer that prompts login, disable input box without auth 2024-12-12 14:57:52 -08:00
Debanjum
2db7a1ca6b Restart code sandbox on crash in eval github workflow (#1007) 
See
e3fed3750b
for corresponding change to use pm2 to auto-restart code sandbox
 2024-12-12 14:32:03 -08:00
Debanjum
12c976dcb2 Track usage costs from DeepInfra OpenAI compatible API 2024-12-12 14:20:34 -08:00
Debanjum
b0abec39d5 Use chat model name var name consistently and type openai chat utils 2024-12-12 14:20:34 -08:00
Debanjum
4915be0301 Fix initializing chat model names parameter after field rename in #1003 2024-12-12 14:20:33 -08:00
sabaimran
a7d0ed8670 Add carousel for navigating images in the sign up modal 2024-12-12 11:47:41 -08:00
Debanjum
9eb863e964 Restart code sandbox on crash in eval github workflow 2024-12-12 11:28:54 -08:00
Debanjum
01bc6d35dc Rename Chat Model Options table to Chat Model as short & readable (#1003) 
- Previous was incorrectly plural but was defining only a single model
- Rename chat model table field to name
- Update documentation
- Update references functions and variables to match new name
 2024-12-12 11:24:16 -08:00
sabaimran
943065b7b3 Remove dead dependencies and improve the google sign in button 2024-12-12 11:19:19 -08:00
sabaimran
41bb1e60d0 Use the LoginPrompt in the chat history side panel 2024-12-11 22:56:03 -08:00
sabaimran
b60b750555 Update the styling to align with Google branding via the sign in button 
- Disable the gsi client side code since it's being finnicky and inconsistent
 2024-12-11 22:49:11 -08:00
sabaimran
0f8b055b42 Improve padding for space, esp in mobile 2024-12-11 18:22:48 -08:00
sabaimran
142239d2c9 Add mobile friendliness and replace the login page redirects 2024-12-11 18:01:04 -08:00
sabaimran
de6ed2352a Break up the parts of the login dialog into smaller modules to extend for mobile 2024-12-11 17:18:43 -08:00
sabaimran
d35db99c6f Initial version of a carousel working for sign in with steps for email sign up 
- Google sign in is pending with the gsi client code. Will see if I can get that working
- Check in relevant image assets
 2024-12-11 16:54:06 -08:00
aditya218
9be26e1bd2 Fix documentation to point to local environment image_generation.md (#1005) 
Fix documentation to point to local environment.
 2024-12-11 16:12:25 -08:00
sabaimran
530b44cf56 Merge branch 'master' of github.com:khoj-ai/khoj into features/new-sign-in-page 2024-12-11 10:30:13 -08:00
Debanjum
fe09df66b7 Make code sandbox container url accessible to Khoj container in docker compose 2024-12-11 01:14:26 -08:00
Debanjum
59008ae90e Use buildx to create multi platform docker image 2024-12-11 00:21:29 -08:00
Debanjum
ec797bc6b8 Build docker imgs on native arch runners to avoid manifest list error 
This also avoids the need to use --amend and annotate steps when
creating the multi-arch docker images
 2024-12-10 23:16:36 -08:00
Debanjum
5f7b13df2d Fix new docker tags in workflow to not include forward slashes 2024-12-10 22:55:33 -08:00
Debanjum
ba6237b5c0 Fix to create multi-arch builds. Stop docker image overwrites in workflow 2024-12-10 21:08:17 -08:00
sabaimran
44ede26e67 Temporarily disable cloud arm builds while we disambiguate the build issues 2024-12-10 20:00:59 -08:00
Debanjum
33a5efaf4b Fix undefined variable exception during openai provider setup on init 
Resolves #1001
 2024-12-10 19:54:00 -08:00
sabaimran
e43341fdcc Release Khoj version 1.31.0 2024-12-10 19:41:31 -08:00
Debanjum
a757ecfd2b Put the generated assets message after the user query and fix prompt 2024-12-10 19:40:13 -08:00
Debanjum
40e4f2ec2e Reduce clutter in chat message ux on Obsidian 
- Move khoj message border to left like in web ui
- Remove user, sender emojis and name
- Ensure conversation title always at top of chat sessions view,
  even if no chat sessions loaded yet, instead of causing layout shift
  on chat sessions load
 2024-12-10 19:34:17 -08:00
sabaimran
1ec1eff57e Improve mobile header, reduce title bar padding and add conv title 2024-12-10 19:03:57 -08:00
sabaimran
321eeeaed7 Fix setting title of shared conversation, move shared button into the title pane 2024-12-10 18:19:46 -08:00
sabaimran
d7e5a76ace Add an icon in the input bar for research mode 2024-12-10 17:49:25 -08:00
sabaimran
01d000e570 Merge pull request #1002 from khoj-ai/features/improve-multiple-output-mode-generation 
Improve handling of multiple output modes

- Use the generated descriptions / inferred queries to supply context to the model about what it's created and give a richer response
- Stop sending the generated image in user message. This seemed to be confusing the model more than helping.
- Collect generated assets in a structured objects to provide model context. This seems to help it follow instructions and separate responsibility better
- Also, rename the open ai converse method to converse_openai to follow patterns with other providers
 2024-12-10 17:06:19 -08:00
sabaimran
2bb14c55a6 Merge branch 'master' of github.com:khoj-ai/khoj into features/improve-multiple-output-mode-generation 2024-12-10 16:56:36 -08:00
sabaimran
6c8007e23b Improve handling of multiple output modes 
- Use the generated descriptions / inferred queries to supply context to the model about what it's created and give a richer response
- Stop sending the generated image in user message. This seemed to be confusing the model more than helping.
- Also, rename the open ai converse method to converse_openai to follow patterns with other providers
 2024-12-10 16:54:21 -08:00
Debanjum
4bc5c1357a Upgrade server, documentation dependencies. Spell fix docker-compose.yml 2024-12-10 15:47:47 -08:00
Debanjum
f8957e52bf Keep chatml message content simple for wider compat unless attachments 
This allows for wider compatibility with chat models and openai proxy
ai model apis that expect message content to be string format, not
objects.
 2024-12-10 00:10:56 -08:00
sabaimran
4b4e0e20d4 Make the version number a badge, rather than an independent item in the nav dropdown 2024-12-09 14:45:26 -08:00
sabaimran
eb36492ba5 Update handling of images when included in the chat history with assistant message 2024-12-08 21:46:07 -08:00
Debanjum
b660c494bc Use recognizable DB model names to ease selection UX on Admin Panel 
Previously id were used (by default) for model display strings.
This made it hard to select chat model options, server chat settings
etc. in the admin panel dropdowns.

This change uses more recognizable names for the DB objects to ease
selection in dropdowns and display in general on the admin panel.
 2024-12-08 20:34:50 -08:00
Debanjum
d10dc9cfe1 Inform code tool AI only limited python packages are available to it 
Reduce code tool failing with module not found errors
 2024-12-08 20:34:50 -08:00
Debanjum
3fd8614a4b Only auto load available chat models from Ollama provider for now 
Allowing models from any openai proxy service makes it too unwieldy.
And a bunch of them do not even support this endpoint.
 2024-12-08 20:34:50 -08:00
sabaimran
2c934162d3 Add a data filter for privacy_level of agents 2024-12-08 19:55:57 -08:00
sabaimran
3b9f4c4356 Correct negative for running prod image locally 2024-12-08 19:55:35 -08:00
Debanjum
9dd3782f5c Rename OpenAIProcessorConversationConfig DB model to more apt AiModelApi (#998) 
* Rename OpenAIProcessorConversationConfig to more apt AiModelAPI

The DB model name had drifted from what it is being used for,
a general chat api provider that supports other chat api providers like
anthropic and google chat models apart from openai based chat models.

This change renames the DB model and updates the docs to remove this
confusion.

Using Ai Model Api we catch most use-cases including chat, stt, image generation etc.
 2024-12-08 18:02:29 -08:00
sabaimran
df66fb23ab Centralize definition of the content security policy and add in-app chat 
- in-app chat is meant for support requests and currently is only in the settings page, where users are most likely to go if confused IMO
 2024-12-08 17:57:27 -08:00
sabaimran
0b87c13f8d Add khoj_version to the settings menu 2024-12-08 17:55:56 -08:00
sabaimran
47a087c73b Fix chatwoot import issue by checking whether we're in an execution environment before loading the script 2024-12-08 17:16:20 -08:00
sabaimran
66f59c8d41 Add Chatwoot to documentation 
See repo: https://github.com/chatwoot/chatwoot
 2024-12-08 16:51:43 -08:00
sabaimran
6ed051d631 Merge pull request #994 from khoj-ai/features/update-desktop-app 
Simplify the desktop app

- Make the desktop app mainly a file-syncing client for users who have lots of documents that they need to share with Khoj. This is because the web app provides a fairly robust chat client which can be used by anyone on their computer.
- The chat client in the desktop app had significantly drifted from our current brand / them, and didn't provide enough value add to update. Later, we will make it easier to install the existing web app as a desktop PWA.
 2024-12-08 15:05:35 -08:00
sabaimran
05b3911080 Update some button titles and add descriptions for clarity 2024-12-08 14:29:12 -08:00
sabaimran
b78b92d6a0 Merge branch 'master' of github.com:khoj-ai/khoj into features/update-desktop-app 2024-12-08 14:20:20 -08:00
sabaimran
e3789aef49 Merge pull request #992 from khoj-ai/features/allow-multi-outputs-in-chat 
Currently, Khoj has terminal states with respect to what assets it outputs. We limit it to image, text, and excalidraw diagram. This limitation is unnecessary and provides undue constraints for creating more dynamic, diverse experiences. For instance, we may want the chat view to morph for document editing or generation, in which case having limited output modes would be a detriment.

This change allows us to emit generated assets and then continue on to more text generation in final response. It forces text response for all messages. It adds a new stream event, GENERATED_ASSETS, which holds content that the AI is emitting in response to the query.
 2024-12-08 14:19:05 -08:00
sabaimran
a2251f01eb Make result optional for code context, relevant when code execution was unsuccessful 2024-12-08 13:27:33 -08:00
sabaimran
9c403d24e1 Fix reference to directory in the eval workflow for starting terrarium 2024-12-08 13:03:05 -08:00
sabaimran
7cd2855146 Make attributes optional in the knowledge graph model 2024-12-08 12:23:17 -08:00
sabaimran
2af687d1c5 Allow snippetHighlighted to also be nullable 2024-12-08 11:51:24 -08:00
sabaimran
efa23a8ad8 Update validation requirements for online searches 2024-12-08 11:30:17 -08:00
sabaimran
6940c6379b Add sudo when running installations in order to install relevant packages 
add --legacy-peer-deps temporarily to see if it helps mitigate the issue
 2024-12-08 11:11:13 -08:00
sabaimran
4c4b7120c6 Use Khoj terrarium fork instead of building from official Cohere repo 2024-12-08 11:06:33 -08:00
sabaimran
a138845fea Merge branch 'master' of github.com:khoj-ai/khoj into features/allow-multi-outputs-in-chat 2024-12-08 10:57:16 -08:00
sabaimran
19832a3ed0 Add note to uncomment line when using the prod image 2024-12-05 18:16:01 -08:00
sabaimran
110c64ba27 Update the desktop instructions 2024-12-05 17:39:16 -08:00
Debanjum
65c5b163c9 Add khoj_lantern svg to web public assets for use by new admin panel 2024-12-05 10:57:18 -08:00
Debanjum
354dc12b3b Style the Admin Panel with a modern theme and Khoj branding (#999) 
Overview
- The default django admin panel UI looks pretty dated and didn't
  have any Khoj specific branding
- Used the Unfold Django admin panel theme for a modern look
- Used the Khoj logo and name in Admin panel title, headings, favicons

Details:
All models shown on Admin panel need to inherit from unfold's
ModelAdmin to get styling applied. So

- Make all models on Admin panel inherit from unfold's ModelAdmin
- Subclassed UserAdmin to inherit from unfold's ModelAdmin
- Deregistered the unused Auth Group model from the Admin panel
  We can add it back when its actually used. Avoid confusion for now
- Explicitly register DjangoJobExecution on admin panel and again make
  it inherit from the unfold.admin.ModelAdmin
 2024-12-04 23:53:43 -08:00
Matias Forbord
9cc79c0fb7 Fix broken doc links to query filters from emacs docs page (#1000) 
* docs: repair query filters link

* change docs link to be relative
 2024-12-04 23:52:27 -08:00
sabaimran
8953ac03ec Rename additional context for llm response to program execution context 2024-12-04 18:43:41 -08:00
sabaimran
886fe4a0c9 Merge branch 'master' of github.com:khoj-ai/khoj into features/allow-multi-outputs-in-chat 2024-12-03 21:37:00 -08:00
sabaimran
df5e34615a Fix processing of images field when construct chat messages 2024-12-03 21:26:55 -08:00
sabaimran
3552032827 Rename additional context to additional_context_for_llm_response 2024-12-03 21:23:15 -08:00
sabaimran
d507894546 Simplify the desktop app 
- Make the desktop app mainly a file-syncing client for users who have lots of documents that they need to share with Khoj. This is because the web app provides a fairly robust chat client which can be used by anyone on their computer.
- The chat client in the desktop app had significantly drifted from our current brand / them, and didn't provide enough value add to update. Later, we will make it easier to install the existing web app as a desktop PWA.
 2024-12-02 15:54:05 -08:00
Debanjum
9f7cb335c5 Create Android app for Khoj (#991) 
Use bubblewrap to publish the Khoj Progressive Web App (PWA) as a Trusted Web Activity (TWA) to the Android Play Store
 2024-12-02 15:45:04 -08:00
Debanjum
ee28d7f125 Fix Android Studio build warnings by using newer gradle, mavenCentral 2024-12-02 12:48:33 -08:00
Debanjum
5d6fb07066 Fix app icons, orientation. Improve name, id, description in webmanifest 
- Use bubblewrap generated splash screen, notification icons from
  1200x1200 high res khoj icon in assets.khoj.dev.
- Discard bubblewrap generated launcher icons.
- Fix orientation to respect phone orientation settings. "any" was not.
 2024-12-02 12:48:25 -08:00
Debanjum
147c8e9115 Release v3 with high-res splash screen. More details in web, app manifest 
- Add 512, 192 Khoj maskable icons to web app manifest for android rendering
- Add id, categories etc suggested by pwabuilder
- Use higher quality icon images for splash screen than what
  bubblewrap creates by default
 2024-12-02 11:28:35 -08:00
Debanjum
d333e10e64 Encode request params as utf-8 to fix multibyte char error in khoj.el 
Encode api key in header, POST request body and GET query param for
search as utf-8 to avoid the multibyte char in request issue when
making API calls from khoj.el to khoj server.

Resolves #935
 2024-12-02 02:00:14 -08:00
Debanjum
db29894038 Do not wrap filepath in Path to fix indexing markdown files on Windows (#993) 
### Issue
- Path with / are converted to \\ on Windows using the `Path' operator.
- The `markdown_to_entries' module was trying to normalize file paths with`Path'  for some reason.
  This would store the file paths in DB Entry differently than the file to entries map if Khoj ran on Windows.
  That'd result in a KeyError when trying to look up the entry file path from `file_to_text_map' in the `text_to_entries:update_embeddings()' function.

### Fix
- Removing the unnecessary OS dependent Path normalization in `markdown_to_entries' should keep the file path storage consistent across `file_to_text_map' var, `FileObjectAdaptor', `Entry' DB tables on Windows for Markdown files as well.

This issue will affect users hosting Khoj server on Windows and attempting to index markdown files.

Resolves #984
 2024-12-02 01:02:58 -08:00
Debanjum
47c926b0ff Add more typing to org|md_to_entries. Remove redundant f-string wraps 
- Add type hints to improve maintainability of stabilzed indexing code
- It shouldn't be necessary to wrap string variables in an f-string

This change aims to improve code quality. It should not affect
functionality.
 2024-12-01 23:02:52 -08:00
Debanjum
dffdd81345 Do not wrap filepath in Path to fix indexing markdown files on Windows 
Issue
- Path with / are converted to \\ on Windows using the Path operator.
- The markdown to entries method for some reason was doing this.
  This would store the file paths in DB entry differently than the file
  to entries map. Resulting in a KeyError when trying to look up the
  entry file path from file_to_text_map in the
  text_to_entries:update_embeddings() function.

Fix
- Removing the unnecessary OS dependendent Path normalization in
  markdown_to_entries should keep the file path storage consistent
  across file_to_text_map var, FileObjectAdaptor, Entry DB tables on
  Windows for Markdown files as well

This issue would only affect users hosting Khoj server on Windows and
attempting to index markdown files.

Resolves #984
 2024-12-01 23:00:31 -08:00
sabaimran
c87fce5930 Add a migration to use the new image storage format for past conversations 
- Added it to the Django migrations so that it auto-triggers when someone updates their server and starts it up again for the first time. This will require that they update their clients as well in order to view/consume image content.
- Remove server-side references in the code that allow to parse the text-to-image intent as it will no longer be necessary, given the chat logs will be migrated
 2024-12-01 18:35:31 -08:00
Debanjum
9e0a2c7a98 Restrict generated chat title to 200 chars limit allowed for chat slug 2024-11-30 19:12:03 -08:00
Debanjum
8b8e2be82d Only create subscription object when it does not exist for user 
This avoid unnecessarily throwing an internal server error when the
user tries to sign-up using multiple mechanisms (e.g first by email, then
by google oauth)
 2024-11-30 19:08:34 -08:00
Debanjum
fc6be543bd Improve GPQA eval prompt to imrpove parsing answer from Khoj response 2024-11-30 17:21:09 -08:00
sabaimran
00f48dc1e8 If in the new images format, show the response text in obsidian instead of the inferred query 2024-11-30 14:39:51 -08:00
sabaimran
224abd14e0 Only add the image_url to the constructed chat message if it is a url 2024-11-30 14:39:27 -08:00
sabaimran
991577aa17 Allow a None turnId to accommodate historic chat messages 2024-11-30 14:39:08 -08:00
sabaimran
a539761c49 Fix processing of excalidrawdiagram in json response chunking 2024-11-30 12:35:13 -08:00
sabaimran
dc4a9ee3e1 Ensure that the generated assets are maintained in the chat window after streaming is completed. 2024-11-30 12:31:20 -08:00
sabaimran
e3aee50cf3 Fix parsing of generated_asset response 2024-11-29 18:41:53 -08:00
sabaimran
2b32f0e80d Remove commented out code blocks 2024-11-29 18:11:50 -08:00
sabaimran
df855adc98 Update response handling in Obsidian to work with new format 2024-11-29 18:10:47 -08:00
sabaimran
512cf535e0 Collapse train of thought when completed during live stream 2024-11-29 18:10:35 -08:00
sabaimran
a0b00ce4a1 Don't include null attributes when filling in stored conversation metadata 
- Prompt adjustments to indicate to LLM what context it has
 2024-11-29 18:10:14 -08:00
sabaimran
c5329d76ba Merge branch 'master' of github.com:khoj-ai/khoj into features/allow-multi-outputs-in-chat 2024-11-29 14:12:03 -08:00
sabaimran
46f647d91d Improve image rendering for khoj generated images. FIx typing of stored excalidraw image. 2024-11-29 14:11:48 -08:00
Debanjum
fdf69b7049 Publish second version with new upload key 2024-11-28 22:04:10 -08:00
Debanjum
faf15072b6 Create first version of Khoj Android app from PWA using Bubblewrap 2024-11-28 22:04:10 -08:00
sabaimran
4f6d1211ba Fix additional context type in anthropic chat 2024-11-28 20:16:36 -08:00
sabaimran
6f408948d3 Fix typing of generated_fiels parameters 2024-11-28 20:15:10 -08:00
sabaimran
439b18c21f Release Khoj version 1.30.10 2024-11-28 19:43:06 -08:00
sabaimran
2dfd163430 Add more explicity run strategies in the runner matrix 2024-11-28 19:31:34 -08:00
sabaimran
80cd902c86 Since linux/amd64 images aren't being created, try setting a custom description on the image 
Refer to this GH documentation on working with multi arch images in the container registry:
https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#adding-a-description-to-multi-arch-images
 2024-11-28 19:14:06 -08:00
sabaimran
40d8a7a581 Release Khoj version 1.30.9 2024-11-28 18:45:50 -08:00
sabaimran
87aa653c7f Add additional steps in prod.Dockerfile to ensure dependencies are copied over 2024-11-28 18:37:08 -08:00
sabaimran
d91935c880 Initial commit of a functional but not yet elegant prototype for this concept 2024-11-28 17:28:23 -08:00
Debanjum
a552543f4f Use json5 to parse llm generated questions to query docs and web 
json5 is more forgiving, handles double quotes, newlines in raw json
string
 2024-11-28 14:35:34 -08:00
Debanjum
0a69af4f61 Update to latest ToDesktop runtime 2024-11-28 13:56:14 -08:00
Debanjum
1d0fe141dc Release Khoj version 1.30.8 2024-11-28 13:37:30 -08:00
Debanjum
29e801c381 Add MATH500 dataset to eval 
Evaluate simpler MATH500 responses with gemini 1.5 flash

This improves both the speed and cost of running this eval
 2024-11-28 12:48:25 -08:00
Debanjum
22aef9bf53 Add GPQA (diamond) dataset to eval 2024-11-28 12:48:25 -08:00
Debanjum
f1190ccf32 Improve parsing complex json strings returned by LLM (#989) 
- Improve escaping to load complex json objects
- Fallback to a more forgiving [json5](https://json5.org/) loader if `json.loads` cannot parse complex json str

This should reduce failures to pick research tool and run code by agent
 2024-11-28 11:01:39 -08:00
Debanjum
8c120a5139 Fallback to json5 loader if json.loads cannot parse complex json str 
JSON5 spec is more flexible, try to load using a fast json5 parser if
the stricter json.loads from the standard library can't load the
raw complex json string into a python dictionary/list
 2024-11-26 21:17:00 -08:00
Debanjum
70b7e7c73a Improve load of complex json objects. Use it to pick tool, run code 
Gemini doesn't work well when trying to output json objects. Using it
to output raw json strings with complex, multi-line structures
requires more intense clean-up of raw json string for parsing
 2024-11-26 17:37:57 -08:00
Debanjum
8cb0db0051 Fix llama-cpp-python install by pytest github workflow 
- Use pre-built wheels for torch and llama-cpp-python
- Install and link musl as it's used by llama-cpp-python pre-built
  wheel instead of glibc
- Join Install git and Install Dependencies steps in pytest workflow
  To remove unnecessary steps
 2024-11-26 02:04:36 -08:00
Debanjum
29315f44e7 Add assetlinks.json to link android app to app.khoj.dev domain 
Add sha cert of android upload, signing keys to open debug, prod apps
as TWA in fullscreen on android phones
 2024-11-26 01:57:54 -08:00
Debanjum
a97a45bf20 Align agent personality with recently updated khoj personality 
See update to Khoj personality in commit
6eb59464da
 2024-11-26 00:06:16 -08:00
Debanjum
e088fcbc7b Build for arm64 on arm64 runner. Parallelize arm64, x64 docker builds 
- Building arm64 image on an ubuntu arm64 runner reduces `yarn build'
  step time by 75% from 12mins to 3mins.
  - This is because no QEMU emulation for arm64 on x86 is required now
- Parallelizing x64 and arm64 platform builds halves build time on top
  - Revert to use standard ubuntu-latest runner as large x64 runner
    doesn't give much more speed improvements

This results an effective additional 50%-66% reduction in build time
on top of #987.

So a full dockerize workflow run now takes *10 mins* vs previous 35+mins.
This is a total of *72% improvement* in max dockerize run time.

Get additional speed improvements when docker layer cache hit.
 2024-11-24 23:18:55 -08:00
Debanjum
5723a3778e Speed up Docker image builds using multi-stage parallel pipelines (#987) 
## Objective
Improve build speed and size of khoj docker images

## Changes
### Improve docker image build speeds
  - Decouple web app and server build steps
  - Build the web app and server in parallel
  - Cache docker layers for reuse across dockerize github workflow runs
    - Split Docker build layers for improved cacheability (e.g separate `yarn install` and `yarn build` steps)
### Reduce size of khoj docker images 
  - Use an up-to-date `.dockerignore` to exclude unnecessary directories
  - Do not installing cuda python packages for cpu builds
### Improve web app builds
  - Use consistent mechanism to get fonts for web app
  - Make tailwind extensions production instead of dev dependencies
  - Make next.js create production builds for the web app (via `NODE_ENV=production` env var)
 2024-11-24 21:49:46 -08:00
Debanjum
4a5646c8da Cache docker layers, nextjs builds in dockerize github workflow 2024-11-24 21:06:22 -08:00
Debanjum
6a39651ad3 Standardize loading fonts locally across pages on web app 2024-11-24 20:41:15 -08:00
sabaimran
9368699b2c Migrate the pre-commit config 2024-11-24 14:54:26 -08:00
sabaimran
6eb59464da Add additional reinforcement to coax gemini into giving a minimum helpful response 2024-11-24 14:53:53 -08:00
sabaimran
15f062b34a Remove print statement for agent style map 2024-11-24 14:53:53 -08:00
sabaimran
d7e68a2d1b Wait for iplcodata to load before first message 
- Fix the console khoj ai ascii art
- Remove some not so good suggested prompt
 2024-11-24 14:53:53 -08:00
Debanjum
f51e0f7859 Make Next.js create production builds of web app for Docker images 2024-11-24 13:59:40 -08:00
Debanjum
710e00ad9e Make tailwind extensions prod, instead of dev, deps of web app 2024-11-24 13:59:40 -08:00
Debanjum
4b486ea5f6 Exclude unnecessary directories from final docker builds 2024-11-24 13:59:40 -08:00
Debanjum
78d8ca49ec Skip Nvidia GPU python packages during Server install in Dockerfiles 2024-11-24 13:59:39 -08:00
Debanjum
37887a175a Speed up Docker image builds using multi-stage parallel pipelines 
Decouple web app, server builds in parallel to speed up Docker
builds
 2024-11-24 12:48:30 -08:00
Debanjum
7c77d65d35 Improve logic to disable telemetry via KHOJ_TELEMETRY_DISABLE env var 
The newly added KHOJ_TELEMETRY_DISABLE env var knob to disable
telemetry should override old config mechanism when set
 2024-11-24 00:54:16 -08:00
sabaimran
2d683898c2 Release Khoj version 1.30.7 2024-11-23 22:51:10 -08:00
sabaimran
914ff994f7 Fix cost addition to chat_metadata 2024-11-23 22:50:45 -08:00
Debanjum
caaa127dcf Release Khoj version 1.30.6 2024-11-23 21:07:00 -08:00
Debanjum
57b8273002 Fix apt install for musl-dev in prod.Dockerfile 2024-11-23 21:06:09 -08:00
Debanjum
8f966b11ec Release Khoj version 1.30.5 2024-11-23 20:49:05 -08:00
Debanjum
498895a47d Fix libmusl error using pre-built llama-cpp-python wheel in prod Docker 2024-11-23 20:47:41 -08:00
Debanjum
e5b211a743 Release Khoj version 1.30.4 2024-11-23 19:48:21 -08:00
Debanjum
9848d89d03 Try build docker images with github high cpu, ram runner 2024-11-23 19:09:36 -08:00
Debanjum
04bb3d6f15 Fix libmusl error using pre-built llama-cpp-python wheel via Docker 
Seems like llama-cpp-python pre-built wheels need libmusl. Otherwise
you run into runtime errors on Khoj startup via Docker.
 2024-11-23 18:46:44 -08:00
Debanjum
8dd2122817 Set sample size to 200 for automated eval runs as well 2024-11-23 14:48:38 -08:00
Debanjum
c4ef31d86f Release Khoj version 1.30.3 2024-11-23 14:40:06 -08:00
Debanjum
15ae22bdcf Use pre-built llama-cpp-python wheel in Khoj docker images 
Reduces build time and resolves FileNotFoundError 'ninja' during
llama-cpp-python local build.
 2024-11-23 14:38:07 -08:00
sabaimran
4ac49ca90f Release Khoj version 1.30.2 2024-11-23 12:00:28 -08:00
sabaimran
eb1b21baaa Add a new sign in modal that is triggered from the login prompt screen, rather than redirecting user to another screen to sign in 2024-11-23 11:55:34 -08:00
Debanjum
5aa5cb1941 Add "New" section with latest updates to Readme 2024-11-23 01:36:50 -08:00
sabaimran
7f5bf35806 Disambiguate renewal_date type. Previously, being used as None, False, and Datetime in different places. 2024-11-22 12:06:20 -08:00
sabaimran
5e8c824ecc Improve the experience for finding past conversation 
- add a conversation title search filter, and an agents filter, for finding conversations
- in the chat session api, return relevant agent style data
 2024-11-22 12:03:01 -08:00
sabaimran
a761865724 Fix handling of customer.subscription.updated event to process new renewal end date 2024-11-22 12:03:01 -08:00
sabaimran
6a054d884b Add quicker/easier filtering on auth 2024-11-22 12:03:01 -08:00
Debanjum
b9a889ab69 Fix Khoj responses when code generated charts in response context 
The current fix should improve Khoj responses when charts in response
context. It truncates code context before sharing with response chat actors.

Previously Khoj would respond with it not being able to create chart
but than have a generated chart in it's response in default mode.

The truncate code context was added to research chat actor for
decision making but it wasn't added to conversation response
generation chat actors.

When khoj generated charts with code for its response, the images in
the context would exceed context window limits.

So the truncation logic to drop all past context, including chat
history, context gathered for current response.

This would result in chat response generator 'forgetting' all for the
current response when code generated images, charts in response context.
 2024-11-21 14:43:52 -08:00
Debanjum
5475a262d4 Move truncate code context func for reusability across modules 
It needs to be used across routers and processors. It being in
run_code tool makes it hard to be used in other chat provider contexts
due to circular dependency issues created by
send_message_to_model_wrapper func
 2024-11-21 14:27:39 -08:00
Debanjum
f434c3fab2 Fix toggling prompt tracer on/off in Khoj via PROMPTRACE_DIR env var 
Previous changes to depend on just the PROMPTRACE_DIR env var instead
of KHOJ_DEBUG or verbosity flag was partial/incomplete.

This fix adds all the changes required to only depend on the
PROMPTRACE_DIR env var to enable/disable prompt tracing in Khoj.
 2024-11-21 14:06:00 -08:00
Debanjum
4a40cf79c3 Add docs on how to cross-device access self-hosted khoj using tailscale 2024-11-21 11:07:18 -08:00
Debanjum
1f96c13f72 Enable starting khoj uvicorn server with ssl cert file, key for https 
Pass your domain cert files via the --sslcert, --sslkey cli args.
For example, to start khoj at https://example.com, you'd run command:

KHOJ_DOMAIN=example.com khoj --sslcert example.com.crt --sslkey
example.com.key --host example.com

This sets up ssl certs directly with khoj without requiring a
reverse proxy like nginx to serve khoj behind https endpoint for
simple setups. More complex setups should, of course, still use a
reverse proxy for efficient request processing
 2024-11-21 11:07:18 -08:00
sabaimran
9fea02f20f In telemetry, differentiate create_user google and email 2024-11-21 11:01:37 -08:00
sabaimran
9db885b5f7 Limit access to chat models to futurist users 2024-11-21 07:53:24 -08:00
sabaimran
7a00a07398 Add trailing slash to Ollama url in docs 2024-11-21 07:48:18 -08:00
sabaimran
3519dd76f0 Fix type of excalidraw image response 2024-11-20 19:01:13 -08:00
sabaimran
467de76fc1 Improve the image diagramming prompts and response parsing 2024-11-20 18:59:40 -08:00
Debanjum
50d8405981 Enable khoj to use terrarium code sandbox as tool in eval workflow 2024-11-20 14:19:27 -08:00
Debanjum
2203236e4c Update desktop app dependencies 2024-11-20 13:05:55 -08:00
Debanjum
409204917e Update documentation website dependencies 2024-11-20 13:05:32 -08:00
Debanjum
6f1adcfe67 Track Usage Metrics in Chat API. Track Running Cost, Accuracy in Evals (#985) 
- Track, return cost and usage metrics in chat api response
  Track input, output token usage and cost of interactions with 
  openai, anthropic and google chat models for each call to the khoj chat api
- Collect, display and store costs & accuracy of eval run currently in progress
  This provides more insight into eval runs during execution 
  instead of having to wait until the eval run completes.
 2024-11-20 12:59:44 -08:00
Debanjum
ffbd0ae3a5 Fix eval github workflow to run on releases, i.e on tags push 2024-11-20 12:57:42 -08:00
Debanjum
ed364fa90e Track running costs & accuracy of eval runs in progress 
Collect, display and store running costs & accuracy of eval run.

This provides more insight into eval runs during execution instead of
having to wait until the eval run completes.
 2024-11-20 12:40:51 -08:00
Debanjum
bbd24f1e98 Improve dropdown menus on web app setting page with scroll & min-width 
- Previously when settings list became long the dropdown height would
  overflow screen height. Now it's max height is clamped and  y-scroll
- Previously the dropdown content would take width of content. This
  would mean the menu could sometimes be less wide than the button. It
  felt strange. Now dropdown content is at least width of parent button
 2024-11-20 12:27:13 -08:00
Debanjum
c53c3db96b Track, return cost and usage metrics in chat api response 
- Track input, output token usage and cost for interactions
  via chat api with openai, anthropic and google chat models

- Get usage metadata from OpenAI using stream_options
- Handle openai proxies that do not support passing usage in response

- Add new usage, end response  events returned by chat api.
  - This can be optionally consumed by clients at a later point
  - Update streaming clients to mark message as completed after new
    end response event, not after end llm response event
- Ensure usage data from final response generation step is included
  - Pass usage data after llm response complete. This allows gathering
    token usage and cost for the final response generation step across
    streaming and non-streaming modes
 2024-11-20 12:17:58 -08:00
Debanjum
80df3bb8c4 Enable prompt tracing only when PROMPTRACE_DIR env var set 
Better to decouple prompt tracing from debug mode or verbosity level
and require explicit, independent config to enable prompt tracing
 2024-11-20 11:54:02 -08:00
Debanjum
9ab76ccaf1 Skip adding agent to chat metadata when chat unset to avoids null ref 2024-11-19 21:10:23 -08:00
Debanjum
4da0499cd7 Stream responses by openai's o1 model series, as api now supports it 
Previously o1 models did not support streaming responses via API. Now
they seem to do
 2024-11-19 21:10:23 -08:00
sabaimran
e5347dac8c Fix base image used for prod in docs 2024-11-19 15:51:27 -08:00
sabaimran
b943069577 Fix button text, and login url in self-hosted auth docs 2024-11-19 15:50:13 -08:00
sabaimran
3b5e6a9f4d Update authentication documentation 2024-11-19 15:45:47 -08:00
Debanjum
7bdc9590dd Fix handling sources, output in chat actor when is automated task 
Remove unnecessary ```python prefix removal. It isn't being triggered
in json deserialize path.
 2024-11-19 13:49:27 -08:00
Debanjum
0e7d611a80 Remove ```python codeblock prefix from raw json before deserialize 2024-11-19 12:53:52 -08:00
Debanjum
001c13ef43 Upgrade web app package dependencies 2024-11-19 12:53:52 -08:00
sabaimran
4f5c1eeded Update some of the open graph data for the documentation website 2024-11-19 11:14:46 -08:00
sabaimran
5134d49d71 Release Khoj version 1.30.1 2024-11-18 17:30:33 -08:00
sabaimran
8bdd0b26d3 And a connections clean up decorator to all scheduled tasks 2024-11-18 17:19:36 -08:00
Debanjum
817601872f Update default offline models enabled 2024-11-18 16:38:17 -08:00
Debanjum
45c623f95c Dedupe, organize chat actor, director tests 
- Move Chat actor tests that were previously in chat director tests file
- Dedupe online, offline io selector chat actor tests
 2024-11-18 16:10:50 -08:00
Debanjum
2a76c69d0d Run online, offine chat actor, director tests for any supported provider 
- Previously online chat actors, director tests only worked with openai.
  This change allows running them for any supported onlnie provider
  including Google, Anthropic and Openai.

- Enable online/offline chat actor, director in two ways:
  1. Explicitly setting KHOJ_TEST_CHAT_PROVIDER environment variable to
     google, anthropic, openai, offline
  2. Implicitly by the first API key found from openai, google or anthropic.

- Default offline chat provider to use Llama 3.1 3B for faster, lower
  compute test runs
 2024-11-18 15:11:37 -08:00
Debanjum
653127bf1d Improve data source, output mode selection 
- Set output mode to single string. Specify output schema in prompt
  - Both thesee should encourage model to select only 1 output mode
    instead of encouraging it in prompt too many times
  - Output schema should also improve schema following in general
- Standardize variable, func name of io selector for readability
- Fix chat actors to test the io selector chat actor
- Make chat actor return sources, output separately for better
  disambiguation, at least during tests, for now
 2024-11-18 15:11:37 -08:00
Debanjum
e3fd51d14b Pass user arg to create title from query in new automation flow 2024-11-18 12:58:10 -08:00
Debanjum
9e74de9b4f Improve serializing conversation JSON to print messages on console 
- Handle chatml message.content with non-json serializable data like
  WebP image binary data used by Gemini models
 2024-11-18 12:57:05 -08:00
sabaimran
3f70d2f685 Add more graceful exception handling when tool selection doesn't work 2024-11-18 09:34:49 -08:00
Debanjum
a2ccf6f59f Fix github workflow to start Khoj, connect to PG and upload results 
- Do not trigger tests to run in ci on update to evals
 2024-11-18 04:25:15 -08:00
Debanjum
7c0fd71bfd Add GitHub workflow to quiz Khoj across modes and specified evals (#982) 
- Evaluate khoj on random 200 questions from each of google frames and openai simpleqa benchmarks across *general*, *default* and *research* modes
- Run eval with Gemini 1.5 Flash as test giver and Gemini 1.5 Pro as test evaluator models
- Trigger eval workflow on release or manually
- Make dataset, khoj mode and sample size configurable when triggered via manual workflow
- Enable Web search, webpage read tools during evaluation
 2024-11-18 02:19:30 -08:00
sabaimran
f75085dc7a Release Khoj version 1.30.0 2024-11-17 21:36:22 -08:00
sabaimran
c72813ba67 Merge pull request #981 from rznzippy/bugfix/980/database-connections-leakage 
Fix database connections leakage (#980)
 2024-11-17 21:01:06 -08:00
sabaimran
7d50c6590d Merge pull request #977 from khoj-ai/features/improve-tool-selection 
- JSON extract from LLMs is pretty decent now, so get the input tools and output modes all in one go. It'll help the model think through the full cycle of what it wants to do to handle the request holistically.
- Make slight improvements to tool selection indicators
 2024-11-17 20:08:19 -08:00
sabaimran
282f47e0d6 Add Jina documentation to readme for self-hosting 2024-11-17 17:20:28 -08:00
Debanjum
48567fd468 Do not erase partial message when generation stopped via button on web app 
Previously, we'd replace the generated message with an error message
when message generation stopped via stop button on chat page of web app.
So the partially generated message (which could be useful) gets lost.

This change just stops generation, while keeping the generated
response so any useful information from the partially generated
message can be retrieved.
 2024-11-17 16:29:18 -08:00
Debanjum
285006d6c9 Sync chat models in Khoj with OpenAI proxies (e.g Ollama) on startup 
- Allows managing chat models in the OpenAI proxy service like Ollama.
- Removes need to manually add, remove chat models from Khoj Admin Panel
  for these OpenAI compatible API services when enabled.
- Khoj still mantains the chat models configs within Khoj, so they can
  be configured via the Khoj admin panel as usual.
 2024-11-17 15:34:36 -08:00
Debanjum
4a7f5d1abe Set API keys in docker-compose.yml to enable web search, scrape tools 2024-11-17 15:34:36 -08:00
Debanjum
d6eece63f4 Use Jina API Key of Jina web scraper if configured in DB 
Previously Jina search didn't API key. Now that it does need API key,
we should re-use the API key set in the Jina web scraper config,
otherwise fallback to using JINA_API_KEY from environment variable, if
either is present.

Resolves #978
 2024-11-17 15:34:14 -08:00
sabaimran
6531f24ca0 Further improvements for descriptions to LLM on modes, code, diagram, image. 2024-11-17 13:23:57 -08:00
sabaimran
0eba6ce315 When diagram generation fails, save to conversation log 
- Update tool name when choosing tools to execute
 2024-11-17 13:23:12 -08:00
sabaimran
7e662a05f8 Merge branch 'master' of github.com:khoj-ai/khoj into features/improve-tool-selection 2024-11-17 12:26:55 -08:00
Ilya Khrustalev
00b1af8f99 Fix database connections leakage (#980) 2024-11-17 19:15:05 +01:00
Debanjum
69ef6829c1 Simplify integrating Ollama, OpenAI proxies with Khoj on first run 
- Integrate with Ollama or other openai compatible APIs by simply
  setting `OPENAI_API_BASE' environment variable in docker-compose etc.
- Update docs on integrating with Ollama, openai proxies on first run
- Auto populate all chat models supported by openai compatible APIs
- Auto set vision enabled for all commercial models

- Minor
  - Add huggingface cache to khoj_models volume. This is where chat
  models and (now) sentence transformer models are stored by default
  - Reduce verbosity of yarn install of web app. Otherwise hit docker
  log size limit & stops showing remaining logs after web app install
  - Suggest `ollama pull <model_name>` to start it in background
 2024-11-17 02:08:20 -08:00
Debanjum
2366fa08b9 Update default vision supported & anthropic chat models on first run 
- Update to latest initialize with new claude 3.5 sonnet and haiku models
- Update to set vision enabled for google and anthropic models by
  default. Previously we didn't support but we've supported this for a
  month or two now
 2024-11-17 02:08:20 -08:00
Debanjum
23ab258d78 Improve user conversation config details on Admin panel 
Show user email and chat model that is associated with the user
conversation config
 2024-11-17 02:08:20 -08:00
Debanjum
41d9011a26 Move evaluation script into tests/evals directory 
This should give more space for eval scripts, results and readme
 2024-11-17 02:08:20 -08:00
Debanjum
d9d5884958 Enable evaluating Khoj on the OpenAI SimpleQA bench using eval script 
- Just load the raw csv from OpenAI bucket. Normalize it into FRAMES format
- Improve docstring for frames datasets as well
- Log the load dataset perf timer at info level
 2024-11-17 02:08:20 -08:00
Debanjum
eb5bc6d9eb Remove Talc search bench from Khoj eval script 2024-11-17 02:08:20 -08:00
Debanjum
fc45aceecf Delete unused favicon ico in old web app directory 2024-11-17 02:08:20 -08:00
Debanjum
a16fc3ade8 Only add /research prefix when no slash command in message on web app 
- Explictly adding a slash command is a higher priority intent than
  research mode being enabled in the background. Respect that for a
  more intuitive UX flow.

- Explicit slash commands do not currently work in research mode.
  You've to turn research mode off to use other slash commands. This
  is strange, unnecessary given intent priority is clear.
 2024-11-17 02:08:20 -08:00
sabaimran
a1b4587b34 Remove extract_images flag from PDF loader 2024-11-15 21:46:35 -08:00
sabaimran
15b4cec1e8 Add documentation for how to use the text to image model configs, reduce to Replicate 2024-11-15 15:26:14 -08:00
sabaimran
759873ec44 Add documentation for how to use the text to image model configs 2024-11-15 15:22:06 -08:00
sabaimran
c77dc84a68 Remove output_modes function reference in chat tests 2024-11-15 14:03:07 -08:00
sabaimran
e3f1ea9dee Improve tool, output mode selection process 
- JSON extract from LLMs is pretty decent now, so get the input tools and output modes all in one go. It'll help the model think through the full cycle of what it wants to do to handle the request holistically.
- Make slight improvements to tool selection indicators
 2024-11-15 13:53:53 -08:00
sabaimran
c1a5b32ebf Do not start server when importing the main.py file, unless gunicorn 
- Add more graceful shutdown when closing bg scheduler thread
 2024-11-14 17:36:51 -08:00
sabaimran
be3ee5ec9f Add cool new suggestion cards for math, diagramming 2024-11-14 17:36:51 -08:00
Debanjum
9fc44f1a7f Enable evaluation Khoj on the Talc Search Bench using Eval script 
- Just load the raw jsonl from Github and normalize it into FRAMES format
- Color printed accuracy in eval script to blue for readability
 2024-11-13 22:50:14 -08:00
Debanjum
8e009f48ce Show tool call error in next iteration. Allow rerun if model requests. 
Previously errors would get eaten up but the model wouldn't see
anything. And the model wouldn't be allowed re-run the same query-tool
combination in the next iteration.

This update should give it insight into why it didn't get a result. So
it can make an informed (hopefully better) decision on what to do next.
And re-run the previous query if appropriate.
 2024-11-13 22:50:14 -08:00
Debanjum
604da90fa8 Wrap try/catch around online search in research mode like other tools 
Previously when call to online search API etc. failed, it'd error out
of response to query in research mode. Khoj should skip tool use that
iteration but continue to try respond.
 2024-11-13 16:46:09 -08:00
Debanjum
8851b5f78a Standardize chat message truncation and serialization before print 
Previously chatml messages were just strings, now they can be list of
strings or list of dicts as well.

- Use json seriallization to manage their variations and truncate them
  before printing for context.
- Put logic in single function for use across chat models
 2024-11-13 16:30:17 -08:00
Debanjum
f4e37209a2 Improve error handling, display and configurability of eval script 
- Default to evaluation decision of None when either agent or
  evaluator llm fails. This fixes accuracy calculations on errors
- Fix showing color for decision True
- Enable arg flags to specify output results file paths
 2024-11-13 14:32:22 -08:00
Debanjum
15b0cfa3dd Improve structured message truncation in logger 
Previously chatml messages were just strings.
Since gemini, anthropic models always have messages as list of
strings, truncate those strings instead of the list of message content
 2024-11-13 14:32:22 -08:00
Debanjum
153ae8bea9 Cut binary, long output files from code result for context efficiency 
Removing binary data and truncating large data in output files
generated by code runs should improve speed and cost of research mode
runs with large or binary output files.

Previously binary data in code results was passed around in iteration
context during research mode. This made the context inefficient
because models have limited efficiency and reasoning capabilities over
b64 encoded image (and other binary) data and would hit context limits
leading to unnecessary truncation of other useful context

Also remove image data when logging output of code execution
 2024-11-13 14:32:22 -08:00
sabaimran
de34cc3987 Remove og image url with khoj documentation 2024-11-13 10:23:02 -08:00
sabaimran
4a1b1e8b9a Add support for interrupting messages after they've been sent. 2024-11-12 22:22:45 -08:00
sabaimran
d607ad7a27 Release Khoj version 1.29.1 2024-11-12 10:32:56 -08:00
sabaimran
8ec1764e42 Handle size calculation more gracefully for converted documents, depending on type 2024-11-12 02:00:29 -08:00
sabaimran
b6714c202f Increase the title character limit to 500 for conversations 2024-11-12 01:51:19 -08:00
sabaimran
f05e64cf8c Release Khoj version 1.29.0 2024-11-11 21:46:25 -08:00
sabaimran
47d3c8c235 Remove email query parameter from subscription patch api 2024-11-11 21:39:49 -08:00
sabaimran
d7027109a5 And null handling for response output_files in code output 2024-11-11 21:14:56 -08:00
sabaimran
d68243a3fb Revert clean_json logic temporarily. Eventually, we should do better validation here to extract markdown-formatted json. 2024-11-11 21:05:17 -08:00
sabaimran
1cab6c081f Add better error handling for diagram output, and fix chat history construct 
- Make the `clean_json` method more robust as well
 2024-11-11 20:44:19 -08:00
sabaimran
7bd2f83f97 Wrap test in suggestionCard 2024-11-11 20:12:46 -08:00
Debanjum
48862a8400 Enable Passing External Documents for Analysis in Code Sandbox (#960) 
- Allow passing user files as input into code sandbox for analysis
- Update prompt to give more example of complex, multi-line code
- Simplify logic for model. Run one program at a time, 
  instead of allowing model to run multiple programs in parallel
- Show Code generated charts and docs in Reference pane of web app and make them downloaded
 2024-11-11 19:37:17 -08:00
Debanjum
5078ac0ce2 Await on conversation save when generate conversation title via API 2024-11-11 19:17:39 -08:00
Debanjum
e1d0015248 Allow disabling Khoj telemetry via KHOJ_TELEMETRY_DISABLE env var 2024-11-11 19:17:39 -08:00
Debanjum
a52500d289 Show generated code artifacts before notes and online references 2024-11-11 18:00:22 -08:00
Debanjum
218eed83cd Show output file not code on hover. Remove reference card title border 2024-11-11 18:00:22 -08:00
Debanjum
b970cfd4b3 Align styling of reference panel card across code, docs, web results 
- Add a border below heading
- Show code snippet in pre block
- Overflow-x when reference side panel open to allow seeing whole text
  via x-scroll
- Align header, body position of reference cards with each other
- Only show filename in doc reference cards at message bottom.
  Show full file path in hover and reference side panel
 2024-11-11 18:00:22 -08:00
Debanjum
8e9f4262a9 Render code output files with code references in reference section 
- Improve rendering code reference with better icons, smaller text and
  different line clamps for better visibility
- Show code output files as sub card of code card in reference section
- Allow downloading files generated by code instead of rendering it in
  chat message directly
- Show executed code before online references in reference panel
 2024-11-11 18:00:22 -08:00
Debanjum
92c1efe6ee Fixes to render & save code context with non text based output modes 
- Fix to render code generated chart with images, excalidraw diagrams
- Fix to save code context to chat history in image, diagram output modes
- Fix bug in image markdown being wrapped twice in markdown syntax
- Render newline in code references shown on chat page of web app
  Previously newlines weren't getting rendered. This made the code
  executed by Khoj hard to read in references. This changes fixes that.

  `dangerouslySetInnerHTML' usage is justified as rendered code
  snippet is being sanitized by DOMPurify before rendering.
 2024-11-11 18:00:22 -08:00
Debanjum
af0215765c Decode code text output files from b64 to str to ease client processing 2024-11-11 18:00:22 -08:00
Debanjum
7b39f2014a Enable analysing user documents in code sandbox and other improvements 
- Run one program at a time, instead of allowing model to pass
  multiple programs to run in parallel to simplify logic for model
- Update prompt to give more example of complex, multi-line code
- Allow passing user files as input into code sandbox for analysis

- Log code execution timer at info level to evaluate execution latencies
  in production

- Type the generated code for easier processing by caller functions
 2024-11-11 17:59:37 -08:00
sabaimran
dc109559d4 Research mode gray when off, colored when on 2024-11-11 16:35:07 -08:00
sabaimran
cdda9c2e73 Improve text wrapping for attached files and preview context 
For the research mode toggle, make it not fill when it's off
 2024-11-11 13:32:10 -08:00
sabaimran
dd36303bb7 Fix sending file attachments in save_to_conversation method 
- When files attached but upload fails, don't update the state variables
- Make removing null characters in pdf extraction more space efficient
 2024-11-11 12:53:06 -08:00
Debanjum
ba2471dc02 Do not CRUD on entries, files & conversations in DB for null user (#958) 
Increase defense-in-depth by reducing paths to create, read, update or delete entries, files and conversations in DB when user is unset.
 2024-11-11 12:47:22 -08:00
Debanjum
536fe994be Remove unused db adapter methods, like for fact checker data store 2024-11-11 12:22:34 -08:00
Debanjum
10bca6fa8f Convert required user param check into decorator. Use with more adapters 2024-11-11 12:22:32 -08:00
Debanjum
ff5c10c221 Do not CRUD on entries, files & conversations in DB for null user 
Increase defense-in-depth by reducing paths to create, read, update or
delete entries, files and conversations in DB when user is unset.
 2024-11-11 12:20:07 -08:00
sabaimran
27fa39353e Make custom agent creation flow available to everyone 
- For private agents, add guardrails to prevent against any misuse or violation of terms of service.
 2024-11-11 11:54:59 -08:00
sabaimran
b563f46a2e Merge pull request #957 from khoj-ai/features/include-full-file-in-convo-with-filter 
Support including file attachments in the chat message

Now that models have much larger context windows, we can reasonably include full texts of certain files in the messages. Do this when an explicit file filter is set in a conversation. Do so in a separate user message in order to mitigate any confusion in the operation.

Pipe the relevant attached_files context through all methods calling into models.

This breaks certain prior behaviors. We will no longer automatically be processing/generating embeddings on the backend and adding documents to the "brain". You'll have to go to settings and go through the upload documents flow there in order to add docs to the brain (i.e., have search include them during question / response).
 2024-11-11 11:34:42 -08:00
sabaimran
2bb2ff27a4 Rename attached_files to query_files. Update relevant backend and client-side code. 2024-11-11 11:21:26 -08:00
sabaimran
47937d5148 Merge branch 'features/include-full-file-in-convo-with-filter' of github.com:khoj-ai/khoj into features/include-full-file-in-convo-with-filter 2024-11-11 09:34:08 -08:00
sabaimran
ae4eb96d48 Consolidate file name to icon mapping 2024-11-11 09:34:04 -08:00
Debanjum
7954f39633 Use accept param to file input to indicate supported file types in web app 
Remove unused total size calculations in chat input
 2024-11-11 04:06:17 -08:00
Debanjum
4223b355dc Use python stdlib methods to write pdf, docx to temp files for loaders 
Use python standard method tempfile.NamedTemporaryFile to write,
delete temporary files safely.
 2024-11-11 03:24:50 -08:00
Debanjum
fd15fc1e59 Move construct chat history back to it's original position in file 
Keep function where it original was allows tracking diffs and change
history more easily
 2024-11-11 03:24:50 -08:00
Debanjum
35d6c792e4 Show snippet of truncated messages in debug logs to avoid log flooding 2024-11-11 02:30:38 -08:00
sabaimran
8805e731fd Merge branch 'master' of github.com:khoj-ai/khoj into features/include-full-file-in-convo-with-filter 2024-11-10 19:24:11 -08:00
sabaimran
a5e2b9e745 Exit early when running an automation if the conversation for the automation does not exist. 2024-11-10 19:22:21 -08:00
sabaimran
55200be4fa Apply agent color fill to the toggle both in off and on states 2024-11-10 19:16:43 -08:00
Debanjum
7468f6a6ed Deduplicate online references returned by chat API to clients 
This will ensure only unique online references are shown in all
clients.

The duplication issue was exacerbated in research mode as even with
different online search queries, you can get previously seen results.

This change does a global deduplication across all online results seen
across research iterations before returning them in client reponse.
 2024-11-10 16:10:32 -08:00
Debanjum
137687ee49 Deduplicate searches in normal mode & across research iterations 
- Deduplicate online, doc search queries across research iterations.
  This avoids running previously run online, doc searches again and
  dedupes online, doc context seen by model to generate response.
- Deduplicate online search queries generated by chat model for each
  user query.
- Do not pass online, docs, code context separately when generate
  response in research mode. These are already collected in the meta
  research passed with the user query
- Improve formatting of context passed to generate research response
  - Use xml tags to delimit context. Pass per iteration queries in each
    iteration result
  - Put user query before meta research results in user message passed
    for generating response

This deduplications will improve speed, cost & quality of research mode
 2024-11-10 16:10:32 -08:00
Debanjum
306f7a2132 Show error in picking next tool to researcher llm in next iteration 
Previously the whole research mode response would fail if the pick
next tool call to chat model failed. Now instead of it completely
failing, the researcher actor is told to try again in next iteration.

This allows for a more graceful degradation in answering a research
question even if a (few?) calls to the chat model fail.
 2024-11-10 14:52:02 -08:00
Debanjum
eb492f3025 Only keep webpage content requested, even if Jina API gets more data 
Jina search API returns content of all webpages in search results.
Previously code wouldn't remove content beyond max_webpages_to_read
limit set. Now, webpage content in organic results aree explicitly
removed beyond the requested max_webpage_to_read limit.

This should align behavior of online results from Jina with other
online search providers. And restrict llm context to a reasonable size
when using Jina for online search.
 2024-11-10 14:51:16 -08:00
Debanjum
8ef7892c5e Exclude non-dictionary doc context from chat history sent to chat models 
This fixes chat with old chat sessions. Fixes issue with old Whatsapp
users can't chat with Khoj because chat history doc context was
stored as a list earlier
 2024-11-10 14:51:16 -08:00
Debanjum
d892ab3174 Fix handling of command rate limit and improve rate limit messages 
Command rate limit wouldn't be shown to user as server wouldn't be
able to handle HTTP exception in the middle of streaming.

Catch exception and render it as LLM response message instead for
visibility into command rate limiting to user on client

Log rate limmit messages for all rate limit events on server as info
messages

Convert exception messages into first person responses by Khoj to
prevent breaking the third wall and provide more details on wht
happened and possible ways to resolve them.
 2024-11-10 14:51:16 -08:00
Debanjum
80ee35b9b1 Wrap messages in web, obsidian UI to stay within screen when long links 
Wrap long links etc. in chat messages and train of thought lists on
web app app and obsidian plugin by breaking them into newlines by word
 2024-11-10 14:49:51 -08:00
Debanjum
f967bdf702 Show correct example index being currently processed in frames eval 
Previously the batch start index wasn't being passed so all batches
started in parallel were showing the same processing example index

This change doesn't impact the evaluation itself, just the index shown
of the example currently being evaluated
 2024-11-10 14:49:51 -08:00
Debanjum
84a8088c2b Only evaluate non-empty responses to reduce eval script latency, cost 
Empty responses by Khoj will always be an incorrect response, so no
need to make call to an evaluator agent to check that
 2024-11-10 14:49:51 -08:00
sabaimran
170d959feb Handle offline messages differently, as they don't respond well to the structured messages 2024-11-09 19:52:46 -08:00
sabaimran
2c543bedd7 Add typing to the constructed messages listed 2024-11-09 19:40:27 -08:00
sabaimran
79b15e4594 Only add images when they're present and vision enabled 2024-11-09 19:37:30 -08:00
sabaimran
bd55028115 Fix randint import from random when creating filenames for tmp 2024-11-09 19:17:18 -08:00
sabaimran
92b6b3ef7b Add attached files to latest structured message in chat ml format 2024-11-09 19:17:00 -08:00
sabaimran
835fa80a4b Allow docx conversion in the chatFunction.ts 2024-11-09 18:51:00 -08:00
sabaimran
459318be13 And random suffixes to decreases any clash probability when writing tmp files to disc 2024-11-09 18:46:34 -08:00
sabaimran
dbf0c26247 Remove _summary_ description in function descriptions 2024-11-09 18:42:42 -08:00
sabaimran
e5ac076fc4 Move construct_chat_history method back to conversation.utils.py 2024-11-09 18:27:46 -08:00
sabaimran
bc95a99fb4 Make tracer the last input parameter for all the relevant chat helper methods 2024-11-09 18:22:46 -08:00
sabaimran
ceb29eae74 Add phone number verification and remove telemetry update call from place where authentication middleware isn't yet installed (in the middleware itself). 2024-11-09 12:25:36 -08:00
sabaimran
3badb27744 Remove stored uploaded files after they're processed. 2024-11-08 23:28:02 -08:00
sabaimran
78630603f4 Delete the fact checker application 2024-11-08 17:27:42 -08:00
sabaimran
807687a0ac Automatically generate titles for conversations from history 2024-11-08 16:02:34 -08:00
sabaimran
7159b0b735 Enforce limits on file size when converting to text 2024-11-08 15:27:28 -08:00
sabaimran
4695174149 Add support for file preview in the chat input area (before message sent) 2024-11-08 15:12:48 -08:00
sabaimran
ad46b0e718 Label pages when extract text from pdf, docs content. Fix scroll area in doc preview. 2024-11-08 14:53:20 -08:00
sabaimran
ee062d1c48 Fix parsing for PDFs via content indexing API 2024-11-07 18:17:29 -08:00
sabaimran
623a97a9ee Merge branch 'master' of github.com:khoj-ai/khoj into features/include-full-file-in-convo-with-filter 2024-11-07 17:18:23 -08:00
sabaimran
33498d876b Simplify the share chat page. Don't need it to maintain its own conversation history 
- When chatting on a shared page, fork and redirect to a new conversation page
 2024-11-07 17:14:11 -08:00
sabaimran
4b8be55958 Convert UUID to string when forking a conversation 2024-11-07 17:13:04 -08:00
sabaimran
9bbe27fe36 Set default value of attached files to empty list 2024-11-07 17:12:45 -08:00
sabaimran
3a51996f64 Process attached files in the chat history and add them to the chat message 2024-11-07 16:06:58 -08:00
sabaimran
a89160e2f7 Add support for converting an attached doc and chatting with it 
- Document is first converted in the chatinputarea, then sent to the chat component. From there, it's sent in the chat API body and then processed by the backend
- We couldn't directly use a UploadFile type in the backend API because we'd have to convert the api type to a multipart form. This would require other client side migrations without uniform benefit, which is why we do it in this two-phase process. This also gives us capacity to repurpose the moe generic interface down the road.
 2024-11-07 16:06:37 -08:00
sabaimran
e521853895 Remove unnecessary console.log statements 2024-11-07 16:03:31 -08:00
sabaimran
92c3b9c502 Add function to get an icon from a file type 2024-11-07 16:02:53 -08:00
sabaimran
140c67f6b5 Remove focus ring from the text area component 2024-11-07 16:02:02 -08:00
sabaimran
b8ed98530f Accept attached files in the chat API 
- weave through all subsequent subcalls to models, where relevant, and save to conversation log
 2024-11-07 16:01:48 -08:00
sabaimran
ecc81e06a7 Add separate methods for docx and pdf files to just convert files to raw text, before further processing 2024-11-07 16:01:08 -08:00
sabaimran
394035136d Add an api that gets a document, and converts it to just text 2024-11-07 16:00:10 -08:00
sabaimran
3b1e8462cd Include attach files in calls to extract questions 2024-11-07 15:59:15 -08:00
sabaimran
de73cbc610 Add support for relaying attached files through backend calls to models 2024-11-07 15:58:52 -08:00
Debanjum
4cad96ded6 Add Script to Evaluate Khoj on Google's FRAMES benchmark (#955) 
- Why
We need better, automated evals to measure performance shifts of Khoj
across prompt, model and capability changes.

Google's FRAMES benchmark evaluates multi-step retrieval and reasoning
capabilities of AI agents. It's a good starter benchmark to evaluate Khoj.

- Details
This PR adds an eval script to evaluate Khoj responses on the the FRAMES
benchmark prompts against the ground truth provided by it.

Script allows configuring sample size, batch size, sampling queries from the
eval dataset.

Gemini is used as an LLM Judge to auto grade Khoj responses vs ground truth 
data from the benchmark.
 2024-11-06 17:52:01 -08:00
Debanjum
8679294bed Remove need to set server chat settings from use openai proxies docs 
This was previously required, but now it's only usefuly for more
advanced settings, not typical for self-hosting users.

With recent updates, the user's selected chat model is used for both
Khoj's train of thought and response. This makes it easy to
switch your preferred chat model directly from the user settings
page and not have to update this in the admin panel as well.

Reflect these code changse in the docs, by removing the unnecessary
step for self-hosted users to create a server chat setting when using
an OpenAI proxy service like Ollama, LiteLLM etc.
 2024-11-05 17:10:53 -08:00
Debanjum
05a93fcbed v-align attach, send buttons with chat input text area on web app 
Otherwise, those buttons look off-center when images are attached to
the chat input area
 2024-11-05 17:10:53 -08:00
sabaimran
a0480d5f6c use fill weight for the toggle right (enabled state) for research mode 2024-11-04 22:01:09 -08:00
sabaimran
dc26da0a12 Add uploaded files in the conversation file filter for a new convo 2024-11-04 22:00:47 -08:00
Debanjum
b51ee644aa Fix escaping filename when normalizing in org node parser 2024-11-04 20:24:57 -08:00
Debanjum
5724d16a6f Fix passing images to anthropic chat models to extract questions 2024-11-04 20:24:57 -08:00
sabaimran
cf0bcec0e7 Revert SKIP_TESTS flag in offline chat director tests 2024-11-04 19:06:54 -08:00
sabaimran
1f372bf2b1 Update file summarization unit tests now that multiple files are allowed 2024-11-04 17:45:54 -08:00
sabaimran
7543360210 Merge branch 'master' of github.com:khoj-ai/khoj into features/include-full-file-in-convo-with-filter 2024-11-04 16:55:48 -08:00
sabaimran
b6145df3be Handle file retrieval when agent is None 2024-11-04 16:55:22 -08:00
sabaimran
3dc9139cee Add additional handling for when file_object comes back empty 2024-11-04 16:53:07 -08:00
sabaimran
a27b8d3e54 Remove summarize condition for only 1 file filter 2024-11-04 16:51:37 -08:00
sabaimran
362bdebd02 Add methods for reading full files by name and including context 
Now that models have much larger context windows, we can reasonably include full texts of certain files in the messages. Do this when an explicit file filter is set in a conversation. Do so in a separate user message in order to mitigate any confusion in the operation.

Pipe the relevant attached_files context through all methods calling into models.

We'll want to limit the file sizes for which this is used and provide more helpful UI indicators that this sort of behavior is taking place.
 2024-11-04 16:37:13 -08:00
sabaimran
e3ca52b7cb Use .get() to get text accompanying image url, instead of subindexing 2024-11-04 16:09:16 -08:00
sabaimran
1e89baca7b Deprecate the UserSearchModelConfig and remove all references 
- The server has moved to a model of standardization for the embeddings generation workflow. Remove references to the support for differentiated models.
- The migration script fo ra new model needs to be updated to accommodate full regeneration.
 2024-11-04 12:24:41 -08:00
Debanjum
1ccbf72752 Use logger instead of print to track eval 2024-11-04 00:40:26 -08:00
sabaimran
99c1d2831a Release Khoj version 1.28.3 2024-11-02 12:23:11 -07:00
sabaimran
075b4ecf15 Call subscription_to_state with sync_to_async wrapper when getting user subscription state 
- This is needed in case the renewal_date is not set and we need to reset it for the user
 2024-11-02 12:22:35 -07:00
sabaimran
ec44cbe1e7 Release Khoj version 1.28.2 2024-11-02 07:53:51 -07:00
Debanjum
791eb205f6 Run prompt batches in parallel for faster eval runs 2024-11-02 04:58:03 -07:00
Debanjum
96904e0769 Add script to evaluate khoj on Google's FRAMES benchmark 
Google's FRAMES benchmark evaluates multi-step retrieval and reasoning
capabilities of an agent.

The script uses Gemini as an LLM Judge to evaluate Khoj responses to
the FRAMES benchmark prompts against the ground truth provided by it.
 2024-11-02 04:57:42 -07:00
Debanjum
31b5fde163 Only enable prompt tracer if git python is installed 2024-11-02 02:07:02 -07:00
sabaimran
5b18dc96e0 Release Khoj version 1.28.1 2024-11-01 22:51:51 -07:00
sabaimran
8d1b1bc78e Move the git python dependency into top level dependencies 2024-11-01 22:51:00 -07:00
Debanjum
e85dd59295 Release Khoj version 1.28.0 2024-11-01 19:06:59 -07:00
Debanjum
1f79a10541 Fix link to code execution feature in docs 2024-11-01 18:22:21 -07:00
Debanjum
cff8e02b60 Research Mode [Part 2]: Improve Prompts, Edit Chat Messages. Set LLM Seed for Reproducibility (#954) 
- Improve chat actors and their prompts for research mode.
- Add documentation to enable the code tool when self-hosting Khoj
- Edit Chat Messages
  - Store Turn Id in each chat message. 
  - Expose API to delete chat message.
  - Expose delete chat message button to turn delete chat message from web app
- Set LLM Generation Seed for Reproducible Debugging and Testing
  - Setting seed for LLM generation is supported by Llama.cpp and OpenAI models. 
    This can (somewhat) restrain LLM output
  - Getting fixed responses for fixed inputs helps test, debug longer reasoning chains like used in advanced reasoning
 2024-11-01 18:16:42 -07:00
Debanjum
14e453039d Add prompt tracing, agent personality to infer webpage urls chat actor 2024-11-01 18:12:50 -07:00
Debanjum
ab321dc518 Expect query before tool in response to give think space in research prompt 2024-11-01 17:51:41 -07:00
Debanjum
1a83bbcc94 Clean API chat router. Move FeedbackData response type to router helper 2024-11-01 17:51:41 -07:00
sabaimran
e6eb87bbb5 Merge branch 'improve-debug-reasoning-and-other-misc-fixes' of github.com:khoj-ai/khoj into improve-debug-reasoning-and-other-misc-fixes 2024-11-01 16:48:39 -07:00
sabaimran
a213b593e8 Limit the number of urls the webscraper can extract for scraping 2024-11-01 16:48:36 -07:00
sabaimran
327fcb8f62 create defiltered query after conversation command is extracted 2024-11-01 16:48:03 -07:00
sabaimran
b79a9ec36d Clarify description of the code evaluation environment: not for document creation 2024-11-01 16:47:27 -07:00
Debanjum
9c7b36dc69 Use standard per minute rate limits across user types 2024-11-01 16:16:06 -07:00
Debanjum
ac21b10dd5 Simplify logic to get default search model. Remove unused import 2024-11-01 15:14:00 -07:00
sabaimran
2b35790165 Merge branch 'master' of github.com:khoj-ai/khoj into improve-debug-reasoning-and-other-misc-fixes 2024-11-01 14:51:26 -07:00
Debanjum
22f3ed3f5d Research Mode: Give Khoj the ability to perform more advanced reasoning (#952) 
## Overview
Khoj can now go into research mode and use a python code interpreter. These are experimental features that are being released early for feedback and testing.

- Research mode allows Khoj to dynamically select the tools it needs to best answer the question. It is also allowed more iterations to get to a satisfactory answer. Its more dynamic train of thought is shown to improve visibility into its thinking.
- Adding ability for Khoj to use a python code interpreter is an adjacent capability. It can help Khoj do some data analysis and generate charts for you. A sandboxed python to run code is provided using [cohere-terrarium](https://github.com/cohere-ai/cohere-terrarium?tab=readme-ov-file), [pyodide](https://pyodide.org/).

## Analysis
Research mode (significantly?) improves Khoj's information retrieval for more complex queries requiring multi-step lookups but takes longer to run. It can research for longer, requiring less back-n-forth with the user to find an answer.

Research mode gives most gains when used with more advanced chat models (like o1, 4o, new claude sonnet and gemini-pro-002). Smaller models improve their response quality but tend to get into repetitive loops more often. 

## Next Steps
- Get community feedback on research mode. What works, what fails, what is confusing, what'd be cool to have.
- Tune Khoj's capabilities for longer autonomous runs and to generalize across a larger range of model sizes

## Miscellaneous Improvements
- Khoj's train of thought is saved and shown for all messages, not just the latest one
- Render charts generated by Khoj and code running using the code tool on the web app
- Align chat input color to currently selected agent color
 2024-11-01 14:46:29 -07:00
sabaimran
baa939f4ce When running code, strip any code delimiters. Disable application json type specification in Gemini request. 2024-11-01 13:47:39 -07:00
sabaimran
8fd2fe162f Determine if research mode is enabled by checking the conversation commands and 'linting' them in the selection phase 2024-11-01 13:12:34 -07:00
sabaimran
cead1598b9 Don't reset research mode after completing research execution 2024-11-01 13:00:11 -07:00
Debanjum
c1c779a7ef Do not yaml format raw code results in context for LLM. It's confusing 2024-11-01 12:45:26 -07:00
sabaimran
b3dad1f393 Standardize rate limits to 1/6 ratio 2024-11-01 12:21:09 -07:00
sabaimran
23a49b6b95 Add documentation for python code execution capability 2024-11-01 12:14:33 -07:00
Debanjum
cd75151431 Do not allow auto selecting research mode as tool for now. 
You are required to manually turning it on. This takes longer and
should be a high intent activity initiated by user
 2024-11-01 12:07:52 -07:00
Debanjum
0b0cfb35e6 Simplify in research mode check in api_chat. 
- Dedent code for readability
- Use better name for in research mode check
- Continue to remove inferred summarize command when multiple files in
  file filter even when not in research mode
- Continue to show select information source train of thought.
  It was removed by mistake earlier
 2024-11-01 12:07:08 -07:00
sabaimran
ffa7f95559 Add template for a code sandbox to the docker-compose configuration 2024-11-01 11:50:58 -07:00
Debanjum
73750ef286 Merge branch 'master' into features/advanced-reasoning 2024-11-01 11:42:01 -07:00
sabaimran
1fc280db35 Handle case where infer_webpage_url returns no valid urls 2024-11-01 11:41:32 -07:00
Debanjum
1c920273dd Add Prompt Tracer to Visualize, Analyze and Debug Khoj's Train of Thought (#951) 
## Overview
Use git to capture prompt traces of khoj's train of thought. View, analyze and debug them using your favorite git client (e.g vscode, magit).

- Each commit captures an interaction with an LLM
  The commit writes the query, response and system message each to a separate file in the repo.
  The commit message captures the chat model, Khoj version and other metadata
- Each conversation turn can have multiple interactions with an LLM (e.g Khoj's train of thought)
- Each new conversation turn forks from and merges back into its conversation branch
- Each new conversation branches from the user branch
- Each new user branches from root commit on the main branch

## Usage
1. Set `KHOJ_DEBUG=true` or start khoj in very verbose mode with `khoj -vv` to turn on prompt tracing
2. Chat with Khoj as usual 
3. Open the promptrace git repo to view the generated prompt traces using your favorite git porcelain. 
   The Khoj prompt trace git repo is created at `/tmp/khoj_promptrace` by default. You can configure the prompt trace directory by setting the `PROMPTRACE_DIR`environment variable.

## Implementation
- Add utility functions to capture prompt traces using git (via `gitpython`)
- Make each model provider in Khoj commit their LLM interactions with promptrace
- Weave chat metadata from chat API through all chat actors and commit it to the prompt trace
 2024-11-01 11:33:54 -07:00
sabaimran
33d36ee58c Add experimental notice to research mode tooltip 2024-11-01 11:00:27 -07:00
sabaimran
0145b2a366 Set usage limits on the research mode 2024-11-01 10:29:33 -07:00
sabaimran
3ea94ac972 Only include inferred-queries in chat history when present 2024-10-31 22:01:41 -07:00
sabaimran
149cbe1019 Use bottom anchor for the commandbar popover 2024-10-31 20:40:38 -07:00
sabaimran
21858acccc Remove conversation command always in query, filter out inferred queries that were not with selected tool when going through tool selection iterations 2024-10-31 20:27:38 -07:00
sabaimran
19241805ee Merge branch 'master' of github.com:khoj-ai/khoj into improve-debug-reasoning-and-other-misc-fixes 2024-10-31 18:20:23 -07:00
Debanjum
302bd51d17 Improve online chat actor prompt for research and normal mode 
- Match the online query generator prompt to match the formatting of
  extract questions
- Separate iteration results by newline
- Improve webpage and online tool descriptions
 2024-10-31 18:17:12 -07:00
Debanjum
52163fe299 Improve research planner prompt to reduce looping 2024-10-31 18:17:01 -07:00
sabaimran
7ebf999688 Merge branch 'master' of github.com:khoj-ai/khoj into features/advanced-reasoning 2024-10-31 18:15:13 -07:00
sabaimran
159ea44883 Remove frame references in the diagramming prompts 2024-10-31 18:14:51 -07:00
Debanjum
89597aefe9 Json dump contents in prompt tracer to make structure discernable 2024-10-31 18:08:42 -07:00
Debanjum
5b15176e20 Only add /research prefix in research mode if not already in user query 2024-10-31 18:08:42 -07:00
sabaimran
559601dd0a Do not exit if/else loop in research loop when notes not found 2024-10-31 13:51:10 -07:00
sabaimran
a13760640c Only show trash can when turnId is present 2024-10-31 13:19:16 -07:00
sabaimran
8d1ecb9bd8 Add optional brew steps for docker install 2024-10-31 12:41:53 -07:00
Debanjum
adca6cbe9d Merge branch 'master' into add-prompt-tracer-for-observability 2024-10-31 02:28:34 -07:00
Debanjum
e17dc9f7b5 Put train of thought ui before Khoj response on web app 2024-10-31 02:24:53 -07:00
Debanjum
e8e6ead39f Fix deleting new messages generated after conversation load 2024-10-30 20:56:38 -07:00
Debanjum
cb90abc660 Resolve train of thought component needs unique key id error on web app 2024-10-30 14:00:21 -07:00
Debanjum
ca5a6831b6 Add ability to delete messages from the web app 2024-10-30 14:00:21 -07:00
Debanjum
ba15686682 Store turn id with each chat message. Expose API to delete chat turn 
Each chat turn is a user query, khoj response message pair
 2024-10-30 14:00:21 -07:00
Debanjum
f64f5b3b6e Handle add/delete file filter operation on non-existent conversation 2024-10-30 14:00:21 -07:00
Debanjum
b3a63017b5 Support setting seed for reproducible LLM response generation 
Anthropic models do not support seed. But offline, gemini and openai
models do. Use these to debug and test Khoj via KHOJ_LLM_SEED env var
 2024-10-30 14:00:21 -07:00
Debanjum
d44e68ba01 Improve handling embedding model config from admin interface 
- Allow server to start if loading embedding model fails with an error.
  This allows fixing the embedding model config via admin panel.

  Previously server failed to start if embedding model was configured
  incorrectly. This prevented fixing the model config via admin panel.

- Convert boolean string in config json to actual booleans when passed
  via admin panel as json before passing to model, query configs

- Only create default model if no search model configured by admin.
  Return first created search model if its been configured by admin.
 2024-10-30 14:00:21 -07:00
Debanjum
358a6ce95d Defer turning cursor color to selected agents color for later 
Capability exists but idea needs to be investigated further
 2024-10-30 14:00:21 -07:00
Debanjum
2ac840e3f2 Make cursor in chat input take on selected agent color 2024-10-30 14:00:21 -07:00
Debanjum
1448b8b3fc Use 3rd person for user in research prompt to reduce person confusion 
Models were getting a bit confused about who is search for who's
information. Using third person to explicitly call out on who's behalf
these searches are running seems to perform better across
models (gemini's, gpt etc.), even if the role of the message is user.
 2024-10-30 13:49:48 -07:00
Debanjum
b8c6989677 Separate example from actual question in extract question prompt 2024-10-30 13:49:48 -07:00
Debanjum
86ffd7a7a2 Handle \n, dedupe json cleaning into single function for reusability 
Use placeholder for newline in json object values until json parsed
and values extracted. This is useful when research mode models outputs
multi-line codeblocks in queries etc.
 2024-10-30 13:49:48 -07:00
Debanjum
83ca820abe Encourage Anthropic models to output json object using { prefill 
Anthropic API doesn't have ability to enforce response with valid json
object, unlike all the other model types.

While the model will usually adhere to json output instructions.

This step is meant to more strongly encourage it to just output json
object when response_type of json_object is requested.
 2024-10-30 13:49:48 -07:00
Debanjum
dc8e89b5de Pass tool AIs iteration history as chat history for better context 
Separate conversation history with user from the conversation history
between the tool AIs and the researcher AI.

Tools AIs don't need top level conversation history, that context is
meant for the researcher AI.

The invoked tool AIs need previous attempts at using the tool in this
research runs iteration history to better tune their next run.

Or at least that is the hypothesis to break the models looping.
 2024-10-30 13:49:48 -07:00
Debanjum
d865994062 Rename code tool arg previous_iteration_history' to context' 2024-10-30 13:49:48 -07:00
Debanjum
06aeca2670 Make researcher, docs search AIs ask more diverse retrieval questions 
Models weren't generating a diverse enough set of questions. They'd do
minor variations on the original query. What is required is asking
queries from a bunch of different lenses to retrieve the requisite
information.

This prompt updates shows the AIs the breadth of questions to by
example and instruction. Seem like performance improved based on vibes
 2024-10-30 13:49:48 -07:00
Debanjum
01881dc7a2 Revert "Make extract question prompt in 1st person wrt user as its a user message" 
This reverts commit 6d3602798aa1b95a30c557576fd4f93ddef2ae76.
 2024-10-30 13:49:48 -07:00
Debanjum
3e695df198 Make extract question prompt in 1st person wrt user as its a user message 
Divide Example from Actual chat history section in prompt
 2024-10-30 13:49:48 -07:00
Debanjum
a3751d6a04 Make extract relevant information system prompt work for any document 
Previously it was too strongly tuned for extracting information from
only webpages. This shouldn't be necessary
 2024-10-30 13:49:48 -07:00
Debanjum
a39e747d07 Improve passing user name in pick next research tool prompt 2024-10-30 13:49:48 -07:00
Debanjum
deff512baa Improve research mode prompts to reduce looping, increase webpage reads 2024-10-30 13:49:48 -07:00
Debanjum
d3184ae39a Simplify storing and displaying document results in research mode 
- Mention count of notes and files disovered
- Store query associated with each compiled reference retrieved for
  easier referencing
 2024-10-30 13:49:48 -07:00
Debanjum
8bd94bf855 Do not use a message branch if no msg id provided to prompt tracer 2024-10-30 13:49:48 -07:00
sabaimran
b63fbc5345 Add a simple badget to the dropdown menu that shows subscription status 2024-10-30 13:00:16 -07:00
sabaimran
82f3d79064 Merge branch 'master' of github.com:khoj-ai/khoj into features/advanced-reasoning 2024-10-30 11:32:10 -07:00
sabaimran
2b2564257e Handle subscription case where it's set to trial, but renewal_date is not set. set the renewal_date for LENGTH_OF_FREE_TRIAL days from subscription creation. 2024-10-30 11:05:31 -07:00
Debanjum
9935d4db0b Do not use a message branch if no msg id provided to prompt tracer 2024-10-28 17:50:27 -07:00
Debanjum
d184498038 Pass context in separate message from user query to research chat actor 2024-10-28 15:37:28 -07:00
Debanjum
d75ce4a9e3 Format online, notes, code context with YAML to be legibile for LLM 2024-10-28 15:37:28 -07:00
sabaimran
5bea0c705b Use break-words in the train of thought for better formatting 2024-10-28 15:36:06 -07:00
sabaimran
1f1b182461 Automatically carry over research mode from home page to chat 
- Improve mobile friendliness with new research mode toggle, since chat input area is now taking up more space
- Remove clunky title from the suggestion card
- Fix fk lookup error for agent.creator
 2024-10-28 15:29:24 -07:00
sabaimran
ebaed53069 Merge branch 'master' of github.com:khoj-ai/khoj into features/advanced-reasoning 2024-10-28 12:39:00 -07:00
sabaimran
889dbd738a Add keyword diagram to diagram output mode description 2024-10-28 12:20:46 -07:00
Debanjum
50ffd7f199 Merge branch 'master' into features/advanced-reasoning 2024-10-28 04:10:59 -07:00
Debanjum
a5d0ca6e1c Use selected agent color to theme the chat input area on home page 2024-10-28 03:47:40 -07:00
Debanjum
aad7528d1b Render slash commands popup below chat input text area on home page 2024-10-28 02:06:04 -07:00
Debanjum
3e17ab438a Separate notes, online context from user message sent to chat models (#950) 
Overview
---
- Put context into separate user message before sending to chat model.
  This should improve model response quality and truncation logic in code
- Pass online context from chat history to chat model for response.
  This should improve response speed when previous online context can be reused
- Improve format of notes, online context passed to chat models in prompt.
  This should improve model response quality

Details
---
The document, online search context are now passed as separate user
messages to chat model, instead of being added to the final user message.

This will improve
- Models ability to differentiate data from user query.
  That should improve response quality and reduce prompt injection
  probability
- Make truncation logic simpler and more robust
  When context window hit, can simply pop messages to auto truncate
  context in order of context, user, assistant message for each
  conversation turn in history until reach current user query

  The complex, brittle logic to extract user query from context in
  last user message isn't required.
 2024-10-28 02:03:18 -07:00
Debanjum
8ddd70f3a9 Put context into separate message before sending to offline chat model 
Align context passed to offline chat model with other chat models

- Pass context in separate message for better separation between user
  query and the shared context
- Pass filename in context
- Add online results for webpage conversation command
 2024-10-28 00:22:21 -07:00
Debanjum
ee0789eb3d Mark context messages with user role as context role isn't being used 
Context role was added to allow change message truncation order based
on context role as well.

Revert it for now since currently this is not currently being done.
 2024-10-28 00:04:14 -07:00
Debanjum
4e39088f5b Make agent name in home page carousel not text wrap on mobile 2024-10-27 23:03:53 -07:00
Debanjum
94074b7007 Focus chat input on toggle research mode. v-align it with send button 2024-10-27 22:54:55 -07:00
sabaimran
a691ce4aa6 Batch entries into smaller groups to process 2024-10-27 20:43:41 -07:00
sabaimran
2924909692 Add a research mode toggle to the chat input area 2024-10-27 16:37:40 -07:00
sabaimran
68499e253b Auto-collapse train of thought, show after chat response in history 2024-10-27 15:48:13 -07:00
sabaimran
101ea6efb1 Add research mode as a slash command, remove from default path 2024-10-27 15:47:44 -07:00
sabaimran
0bd78791ca Let user exit from command mode with esc, click out, etc. 2024-10-27 15:01:49 -07:00
sabaimran
a121d67b10 Persist the train of thought in the conversation history 2024-10-26 23:46:15 -07:00
sabaimran
9e8ac7f89e Fix input/output mismatches in the /summarize command 2024-10-26 16:37:58 -07:00
sabaimran
e4285941d1 Use the advanced chat model if the user is subscribed 2024-10-26 16:00:54 -07:00
sabaimran
33e48aa27e Merge branch 'add-prompt-tracer-for-observability' of github.com:khoj-ai/khoj into features/advanced-reasoning 2024-10-26 14:09:00 -07:00
sabaimran
fd71a4b086 Add better exception handling in the prompt trace logic, use default value from parameters 2024-10-26 14:08:00 -07:00
Debanjum
3e5b5ec122 Encourage model to read webpages more often after online search 
Previously model would rarely read webpages after webpage search. Need
the model to webpages more regularly for deeper research and to stop
getting stuck in repetitive online search loops
 2024-10-26 10:49:09 -07:00
Debanjum
bf96d81943 Format online results as YAML to pass it in more readable form to model 
Previous passing of online results as json dump in prompts was less
readable for humans, and I'm guessing less readable for
models (trained on human data) as well?
 2024-10-26 10:49:09 -07:00
Debanjum
3e97ebf0c7 Unescape special characters in prompt traces for better readability 2024-10-26 10:49:09 -07:00
Debanjum
8af9dc3ee1 Unescape special characters in prompt traces for better readability 2024-10-26 10:45:42 -07:00
Debanjum Singh Solanky
0f3927e810 Send gathered references to client after code results calculated 2024-10-26 05:59:10 -07:00
Debanjum Singh Solanky
f04f871a72 Merge branch 'add-prompt-tracer-for-observability' of github.com:khoj-ai/khoj into features/advanced-reasoning 
- Start from this branches src/khoj/routers/api_chat.py
    Add tracer to all old and new chat actors that don't have it set
    when they are called.
  - Update the new chat actors like apick next tool etc to use tracer too
 2024-10-26 05:56:13 -07:00
Debanjum Singh Solanky
ddc6ccde2d Merge branch 'master' into features/advanced-reasoning 
- Conflicts:
  Combine both sides of the conflict in all 3 files below
  - src/khoj/processor/conversation/utils.py
  - src/khoj/routers/helpers.py
  - src/khoj/utils/helpers.py
 2024-10-26 05:15:51 -07:00
Debanjum Singh Solanky
ea0712424b Commit conversation traces using user, chat, message branch hierarchy 
- Message train of thought forks and merges from its conversation branch
- Conversation branches from user branch
- User branches from root commit on the main branch

- Weave chat tracer metadata from api endpoint through all chat actors
  and commit it to the prompt trace
 2024-10-26 05:08:47 -07:00
Debanjum Singh Solanky
a3022b7556 Allow Offline Chat model calling functions to save conversation traces 2024-10-26 05:08:47 -07:00
Debanjum Singh Solanky
eb6424f14d Allow Anthropic API calling functions to save conversation traces 2024-10-26 05:08:47 -07:00
Debanjum Singh Solanky
6fcd6a5659 Allow Gemini API calling functions to save conversation traces 2024-10-26 05:08:47 -07:00
Debanjum Singh Solanky
384f394336 Allow OpenAI API calling functions to save conversation traces 2024-10-26 04:59:21 -07:00
Debanjum Singh Solanky
10c8fd3b2a Save conversation traces to git for visualization 2024-10-26 04:59:19 -07:00
sabaimran
7e0a692d16 Release Khoj version 1.27.1 2024-10-25 15:23:07 -07:00
sabaimran
b257fa1884 Add a None check before doing a DT comparison when getting subscription type 2024-10-25 15:22:48 -07:00
sabaimran
0f6f282c30 Release Khoj version 1.27.0 2024-10-25 14:11:14 -07:00
sabaimran
479e156168 Add to the ConversationCommand.Image description to LLM 2024-10-25 09:14:32 -07:00
sabaimran
a11b5293fb Add uploaded images to research mode, code slash command, include code references 2024-10-24 23:56:24 -07:00
sabaimran
5acf40c440 Clean up summarization code paths 
Use assumption of summarization response being a str
 2024-10-24 23:56:24 -07:00
sabaimran
12b32a3d04 Resolve merge conflicts 2024-10-24 23:43:55 -07:00
Debanjum
adee5a3e20 Give Vision to Anthropic models in Khoj (#948) 
### Major
- Give Vision to Anthropic models in Khoj

### Minor
- Reuse logic to format messages for chat with anthropic models
- Make the get image from url function more versatile and reusable
- Encourage output mode chat actor to output only json and nothing else
 2024-10-24 18:02:38 -07:00
Debanjum Singh Solanky
01d740debd Return typed image from image_with_url function for readability 2024-10-24 17:58:46 -07:00
Debanjum Singh Solanky
37317e321d Dedupe user location passed in image, diagram generation prompts 2024-10-24 01:03:29 -07:00
Debanjum Singh Solanky
2a32836d1a Log more descriptive error when image gen fails with Replicate 2024-10-24 01:03:29 -07:00
sabaimran
30f9225021 Merge branch 'master' of github.com:khoj-ai/khoj into features/advanced-reasoning 2024-10-23 19:15:51 -07:00
sabaimran
5120597d4e Remove user customized search model (#946) 
- Use a single standard search model across the server. There's diminishing benefits for having multiple user-customizable search models. 
- We may want to add server-level customization for specific tasks
- Store the search model used to generate a given entry on the `Entry` object
- Remove user-facing APIs and view
- Add a management command for migrating the default search model on the server

In a future PR (after running the migration), we'll also remove the `UserSearchModelConfig`
 2024-10-23 17:38:37 -07:00
Debanjum Singh Solanky
8d588e0765 Encourage output mode chat actor to output only json and nothing else 
Latest claude model wanted to say more than just give the json output.
The updated prompt encourages the model to ouput just json. This is
similar to what is already being done for other prompts
 2024-10-23 17:19:21 -07:00
Debanjum Singh Solanky
abad5348a0 Give Vision to Anthropic models in Khoj 2024-10-23 17:19:21 -07:00
Debanjum Singh Solanky
6fd50a5956 Reuse logic to format messages for chat with anthropic models 2024-10-23 17:19:21 -07:00
Debanjum Singh Solanky
82eac5a043 Make the get image from url function more versatile and reusable 
It was previously added under the google utils. Now it can be used by
other conversation processors as well.

The updated function
- can get both base64 encoded and PIL formatted images from url
- will return the media type of the image as well in response
 2024-10-23 17:19:20 -07:00
sabaimran
f3ce47b445 Create explicit flow to enable the free trial (#944) 
* Create explicit flow to enable the free trial

The current design is confusing. It obfuscates the fact that the user is on a free trial. This design will make the opt-in explicit and more intuitive.

* Use the Subscription Type enum instead of hardcoded strings everywhere

* Use length of free trial in the frontend code as well
 2024-10-23 15:29:23 -07:00
Debanjum Singh Solanky
bc059eeb0b Merge branch 'master' into put-retrieved-context-in-separate-chatml-message 2024-10-23 12:55:18 -07:00
Debanjum Singh Solanky
3b978b9b67 Fix chat history construction when generating chatml msgs with context 2024-10-23 12:55:12 -07:00
sabaimran
c5e91c346a Fix Docker desktop link for Linux 2024-10-23 11:24:54 -07:00
Debanjum Singh Solanky
9f2c02d9f7 Chat with the default agent by default from web app home 
Had temporarily updated the default selected agent to last used.
Revert for now as
1. The previous logic was buggy. It didn't select the default agent
   even when the last used agent was the default agent. Which would
   require more work.
2. It maybe too early anyway to set the default agent to last used.
 2024-10-23 03:43:57 -07:00
Debanjum Singh Solanky
218946edda Fix copying message with user images on web app 
Adding div elements to message to render degraded text copied to
clipboard for messages with user uploaded images.

This change fixes that by separating message to render from message
for clipboard. It ensures differently formatted forms of the user
images are added to the two to allow proper rendering while still
having decently formatted text copied to clipboard
 2024-10-23 03:41:25 -07:00
Debanjum Singh Solanky
7d9a06c8ab Merge branch 'master' into put-retrieved-context-in-separate-chatml-message 2024-10-23 00:13:38 -07:00
sabaimran
7c29af9745 Add link to self-hosted admin page and add docs for building front-end assets. Close #901 2024-10-22 22:42:27 -07:00
Debanjum Singh Solanky
2a50694089 Allow typing multi-line queries from a phone with Enter key 
Add newline instead of sending message when hit Enter key on mobile
displays. As on phones shift key doesn't exist and send button is easily
clickable.

Limit hitting Enter key to send message to computers = larger display
= expected to have full fledged keyboards.
 2024-10-22 21:20:22 -07:00
Debanjum Singh Solanky
a134cd835c Focus on chat input area to enter text after file uploads on web app 2024-10-22 21:19:17 -07:00
Debanjum
c81e708833 Show all agents, smart sorted, in carousel on home screen of web app (#943) 
## Overview
Allow quickly selecting, switching agents from agents pane on home page of web app

## Details
- Show all agents in carousel on home screen agent pane of web app
- Smart Sort
  1. Pin default agent as first for ease of access
  2. Show used agents by MRU for ease of access
  3. Shuffle unused agents for discoverability
- Select most recently used agent to chat with by default
- Push smart sort logic down to API
  - Common logic can be reused across clients
  - Agent sort was previously done in web app
- Focus on chat input on agent select
- Double click agent on home page to open edit agent card on agents page
 2024-10-22 21:18:17 -07:00
Debanjum Singh Solanky
750fbce0c2 Merge branch 'master' into improve-agent-pane-on-home-screen 2024-10-22 20:05:29 -07:00
Debanjum Singh Solanky
3be505db48 Only show type of error when image generation fails to clients 
Rather than showing raw error message from the underlying service as it
could contain sensitive information
 2024-10-22 20:03:20 -07:00
Debanjum
c6f3253ebd Chat with Multiple Images. Support Vision with Gemini (#942) 
## Overview
- Add vision support for Gemini models in Khoj
- Allow sharing multiple images as part of user query from the web app
- Handle multiple images shared in query to chat API
 2024-10-22 19:59:18 -07:00
Debanjum Singh Solanky
b3fff43542 Sanitize user attached images. Constrain chat input width on home page 
Set max combined images size to 20mb to allow multiple photos to be shared
 2024-10-22 19:42:40 -07:00
Debanjum Singh Solanky
6c393800cc Merge branch 'master' into multi-image-chat-and-vision-for-gemini 2024-10-22 18:38:49 -07:00
Debanjum Singh Solanky
91bbd19333 Close the agent detail hover card when scroll on agent pane 2024-10-22 18:03:17 -07:00
Debanjum Singh Solanky
110c67f083 Improve agent pill, detail card styling. Handle null chatInputRef 
- Remove border from agent detail hover card on home page
- Do not wrap long agent names in agent pills on home page
- Handle scenario where chatInputRef is null
 2024-10-22 18:03:17 -07:00
Debanjum Singh Solanky
aca8bef024 Only use recent chat sessions for agent MRU. Handle null agent chats 2024-10-22 17:46:45 -07:00
sabaimran
0dad4212fa Generate dynamic diagrams (via Excalidraw) (#940) 
Add support for generating dynamic diagrams in flow with Excalidraw (https://github.com/excalidraw/excalidraw). This happens in three steps:
1. Default information collection & intent determination step.
2. Improving the overall guidance of the prompt for generating a JSON, Excalidraw-compatible declaration.
3. Generation of the diagram to output to the final UI.

Add support in the web UI.
 2024-10-22 16:13:46 -07:00
sabaimran
1e993d561b Release Khoj version 1.26.4 2024-10-22 13:50:08 -07:00
Debanjum Singh Solanky
e8fb79a369 Rate limit the count and total size of images shared via API 2024-10-22 04:37:54 -07:00
Debanjum Singh Solanky
39a613d3bc Fix up openai chat actor tests 2024-10-22 03:09:36 -07:00
Debanjum Singh Solanky
0847fb0102 Pass online context from chat history to chat model for response 
Previously only notes context from chat history was included.
This change includes online context from chat history for model to use
for response generation.

This can reduce need for online lookups by reusing previous online
context for faster responses. But will increase overall response time
when not reusing past online context, as faster context buildup per
conversation.

Unsure if inclusion of context is preferrable. If not, both notes and
online context should be removed.
 2024-10-22 03:09:36 -07:00
Debanjum Singh Solanky
0c52a1169a Put context into separate user message before sending to chat model 
The document, online search context are now passed as separate user
messages to chat model, instead of being added to the final user message.

This will improve

- Models ability to differentiate data from user query.
  That should improve response quality and reduce prompt injection
  probability

- Make truncation logic simpler and more robust
  When context window hit, can simply pop messages to auto truncate
  context in order of context, user, assistant message for each
  conversation turn in history until reach current user query

  The complex, brittle logic to extract user query from context in
  last user message isn't required.

Marking the context message with assistant role doesn't translate well
across chat models. E.g
- Gemini can't handle consecutive messages by role = model well
- Claude will merge consecutive messages by same role. In current
  message ordering the context message will result get merged into the
  previous assistant response. And if move context message after user
  query. The truncation logic will have to hop and skip while doing
  deletions
- GPT seems to handle consecutive roles of any type fine

Using context role = user generalizes better across chat models for
now and aligns with previous behavior.
 2024-10-22 03:09:36 -07:00
Debanjum Singh Solanky
7ac241b766 Improve format of notes, online context passed to chat models in prompt 
Improve separation of note snippets and show its origin file in notes
prompt to have more readable, contextualized text shared with model.

Previously the references dict was being directly passed as a string.
The documents don't look well formatted and are less intelligible.

- Passing file path along with notes snippets will help contextualize
  the notes better.
- Better formatting should help with making notes more readable by the
  chat model.
 2024-10-22 03:09:36 -07:00
sabaimran
892040972f Replace user_id with server_id in telemetry 2024-10-21 20:47:52 -07:00
sabaimran
db959a504d Fix the version of pymupdf to avert build errors 2024-10-21 12:56:51 -07:00
sabaimran
21e69b506d Release Khoj version 1.26.3 2024-10-21 08:19:05 -07:00
Debanjum Singh Solanky
9b554feb91 Show agent details card on hover on agent pill on web app home page 
- Double click on agent to open edit agent card
- Focus on chat input pane when agent selected/clicked
  for quick, smooth agent switch and message flow
- Hover on agent to see agent detail card on non-mobile displays
  - Use debounce to only show when hover on card for a bit
 2024-10-21 00:08:01 -07:00
Debanjum Singh Solanky
220ff1df62 Set chatInputArea forward ref from parent components for control 2024-10-21 00:02:48 -07:00
Debanjum Singh Solanky
54b92eaf73 Extract isUserSubscribed check from Agents page to make it resusable 2024-10-20 23:31:48 -07:00
Debanjum Singh Solanky
bdbe8f003e Move agent details and edit card out into reusable components on web app 2024-10-20 23:31:47 -07:00
sabaimran
ad197be70c Fix PDFs unit test, skip OCR 2024-10-20 22:25:41 -07:00
sabaimran
59fec37943 Improve agents management, and limit agents view to private and official agents 
- Default to None for the input_tools and output_modes so that they can be managed in the admin panel
- Hold off on showing off all Public Agents until we have a better experience for user profiles etc.
 2024-10-20 22:24:51 -07:00
sabaimran
a979457442 Add unit tests for agents 
- Add permutations of testing for with, without knowledge base. Private, public, different users.
 2024-10-20 20:04:50 -07:00
Debanjum Singh Solanky
5fca41cc29 Show agents sorted by mru, Select mru agent by default on web app 
Have get agents API return agents ordered intelligently
- Put the default agent first
- Sort used agents by most recently chatted with agent for ease of access
- Randomly shuffle the remaining unused agents for discoverability
 2024-10-20 15:21:25 -07:00
Debanjum Singh Solanky
a6bfdbdbfe Show all agents in carousel on home screen agent pane of web app 
This change wraps the agent pane in a scroll area with all agents shown.
It allows selecting an agent to chat with directly from the home
screen without breaking flow and having to jump to the agents page.

The previous flow was not convenient to quickly and consistently start
chat with one of your standard agents.

This was because a random subet of agents were shown on the home page.
To start chat with an agent not shown on home screen load you had to
open the agents page and initiate the conversation from there.
 2024-10-20 15:21:25 -07:00
Debanjum Singh Solanky
7646ac6779 Style user attached images as carousel on chat input area of web app 2024-10-20 00:40:08 -07:00
sabaimran
5d5bea6a5f Ensure images are reset after messages processed 2024-10-19 22:02:06 -07:00
sabaimran
1ad6e1749f Move window redirect to after relevant data is dropped in localStorage on the homage page 
One limitation of this methodology is that localStorage has a limit in how much data it can take. Should add more graceful error handling here as well.
 2024-10-19 20:36:13 -07:00
sabaimran
cb6b3ec1e9 Improve mode description given to LLM when determining how to respond. 
Currently experiencing difficulty instruction following when an image is shared. It's more likely to try and output an image. Update to make a clearer distinction.
 2024-10-19 20:35:32 -07:00
sabaimran
545259e308 Remove unused icons in chatInputArea 2024-10-19 16:54:21 -07:00
Debanjum Singh Solanky
3cc1426edf Style user attached images with fixed height, in a single row on web app 2024-10-19 16:48:36 -07:00
Debanjum Singh Solanky
58a331227d Display the attached images inside the chat input area on the web app 
- Put the attached images display div inside the same parent div as
  the text area
- Keep the attachment, microphone/send message buttons aligned with
  the text area. So the attached images just show up at the top of the
  text area but everything else stays at the same horizontal height as
  before.

- This improves the UX by
  - Ensuring that the attached images do not obscure the agents pane
    above the chat input area
  - The attached images visually look like they are inside the actual
    input area, rather than floating above it. So the visual aligns
    with the semantics
 2024-10-19 16:29:45 -07:00
Debanjum Singh Solanky
3e39fac455 Add vision support for Gemini models in Khoj 2024-10-19 15:47:03 -07:00
Debanjum Singh Solanky
0d6a54c10f Allow sharing multiple images as part of user query from the web app 
Previously the web app only expected a single image to be shared by
the user as part of their query.

This change allows sharing multiple images from the web app.

Closes #921
 2024-10-19 15:47:03 -07:00
Debanjum Singh Solanky
e2abc1a257 Handle multiple images shared in query to chat API 
Previously Khoj could respond to a single shared image at a time.

This changes updates the chat API to accept multiple images shared by
the user and send it to the appropriate chat actors including the
openai response generation chat actor for getting an image aware
response
 2024-10-19 14:53:33 -07:00
Debanjum Singh Solanky
c6c48cfc18 Fix arg to generate_summary_from_file and type of this_iteration 2024-10-17 13:38:48 -07:00
Debanjum Singh Solanky
feb6d65ef8 Merge branch 'master' into features/advanced-reasoning 2024-10-15 09:37:56 -07:00
Debanjum Singh Solanky
336c6c3689 Show tool to use decision for next iteration in train of thought 2024-10-15 01:12:18 -07:00
Debanjum Singh Solanky
81fb65fa0a Return data sources to use if exception in data source chat actor 
Previously no value was returned if an exception got triggered when
collecting information sources to search.
 2024-10-14 18:20:20 -07:00
Debanjum Singh Solanky
3c93f07b3f Try respond even if web search, webpage read fails during chat 
Khoj shouldn't refuse to respond to user if web lookups fail.
It should transparently mention that online search etc. failed.
But try respond as best as it can without those references

This change ensures a response to the users query is attempted even
when web info retrieval fails.
 2024-10-14 18:13:26 -07:00
Debanjum Singh Solanky
07ab7ebf07 Try respond even if document search via inference endpoint fails 
The huggingface endpoint can be flaky. Khoj shouldn't refuse to
respond to user if document search fails.
It should transparently mention that document lookup failed.
But try respond as best as it can without the document references

This changes provides graceful failover when inference endpoint
requests fail either when encoding query or reranking retrieved docs
 2024-10-14 18:13:26 -07:00
Debanjum Singh Solanky
d6206aa80c Remove deprecated GET chat API endpoint 2024-10-14 18:13:26 -07:00
Debanjum Singh Solanky
263eee4351 Fix default chat model to use user model if no server chat model set 
- Advanced chat model should also fallback to user chat model if set
- Get conversation config should falback to user chat model if set

These assume no server chat model settings is configured
 2024-10-14 18:13:26 -07:00
Debanjum Singh Solanky
abcd11cfc0 Merge branch 'master' into features/advanced-reasoning 2024-10-13 03:06:23 -07:00
Debanjum Singh Solanky
9356e66b94 Fix default chat model to use user model if no server chat model set 
- Advanced chat model should also fallback to user chat model if set
- Get conversation config should falback to user chat model if set

These assume no server chat model settings is configured
 2024-10-13 03:02:29 -07:00
Debanjum Singh Solanky
9314f0a398 Fix default chat configs to use user model if no server chat model set 
Post merge cleanup in advanced reasoning to fallback to user chat
model if no server chat model defined for advanced and default
 2024-10-13 02:59:10 -07:00
Debanjum Singh Solanky
a2200466b7 Merge branch 'master' into features/advanced-reasoning 2024-10-12 21:01:22 -07:00
Debanjum Singh Solanky
9daaae0fdb Render inline any image files output by code in message 
Update regex to also include any links to code generated images that
aren't explicitly meant to be displayed inline. This allows folks to
download the image (unlike the fake link that doesn't work created by
model)
 2024-10-12 10:34:57 -07:00
Debanjum Singh Solanky
20d495c43a Update the iterative chat director prompt to generalize across chat models 
These prompts work across o1 and standard openai model. Works with
anthropic and google models as well
 2024-10-12 10:34:57 -07:00
sabaimran
01a58b71a5 Skip image, code generation if in research mode 2024-10-10 18:06:29 -07:00
Debanjum Singh Solanky
1b13d069f5 Pass data collected from various sources to code tool in normal flow too 2024-10-10 05:19:27 -07:00
Debanjum Singh Solanky
f462d34547 Render images files output by code interpreter in message on web app 2024-10-10 05:17:53 -07:00
Debanjum Singh Solanky
564491e164 Extract date filters quoted with non-ascii quotes in query 2024-10-10 04:45:00 -07:00
Debanjum Singh Solanky
6a8fd9bf33 Reorder embeddings search arguments based on argument importance 2024-10-10 04:45:00 -07:00
Debanjum Singh Solanky
0eacc0b2b0 Use consistent name for user, planner to not miss current user query 
Previously Khoj would start answering the previous query. This maybe
because the prompt uses User for prompt in chat history but was using
Q for current user prompt.
 2024-10-10 04:45:00 -07:00
Debanjum Singh Solanky
284c8c331b Increase default max iterations for research chat director to 5 2024-10-10 04:45:00 -07:00
Debanjum Singh Solanky
1e390325d2 Let research chat director decide which webpage to read, if any 
Make webpages to read automatically on search_online configurable via
a argument.

Set it to default to 1, so other callers of the function
are unaffected.

But iterative chat director can still decide which, if
any, webpages to read based on the online search it performs
 2024-10-10 04:45:00 -07:00
Debanjum Singh Solanky
5a699a52d2 Improve webpage summarization prompt to better extract links, excerpts 
This change allows the iterative director to dive deeper into its
research as the data extracted contains relevant links from the webpage

Previous summarization prompt didn't extract relevant links from the
webpage which limited further explorations from webpages
 2024-10-10 04:45:00 -07:00
Debanjum Singh Solanky
61df1d5db8 Pass previous iteration results to code interpreter chat actors 
This improves the code interpreter chat actors abilitiy to generate
code with data collected during the previous iterations
 2024-10-10 04:45:00 -07:00
Debanjum Singh Solanky
9e7025b330 Set python interpret sandbox url via environment variable 2024-10-10 04:45:00 -07:00
Debanjum Singh Solanky
2dc5804571 Extract defilter query into conversation utils for reuse 2024-10-10 04:45:00 -07:00
sabaimran
e69a8382f2 Add a code icon for code-related train of thought 2024-10-09 23:56:57 -07:00
sabaimran
536422a40c Include code snippets in the reference panel 2024-10-09 23:54:11 -07:00
Debanjum Singh Solanky
8d33c764b7 Allow iterative chat director to use python interpreter as a tool 2024-10-09 23:38:20 -07:00
Debanjum Singh Solanky
b373073f47 Show executed code in web app chat message references 2024-10-09 22:13:18 -07:00
Debanjum Singh Solanky
a98f97ed5e Refactor Run Code tool into separate module and modularize code functions 
Move construct_chat_history and ChatEvent enum into conversation.utils
and move send_message_to_model_wrapper to conversation.helper to
modularize code. And start thinning out the bloated routers.helper

- conversation.util components are shared functions that conversation
  child packages can use.
- conversation.helper components can't be imported by conversation
  packages but it can use these child packages

This division allows better modularity while avoiding circular
import dependencies
 2024-10-09 22:13:17 -07:00
Debanjum Singh Solanky
8044733201 Give Khoj ability to run python code as a tool triggered via chat API 
Create python code executing chat actor
- The chat actor generate python code within sandbox constraints
- Run the generated python code in the cohere terrarium, pyodide
  based sandbox accessible at sandbox url
 2024-10-09 21:37:22 -07:00
Debanjum Singh Solanky
4d33239af6 Improve prompts for the iterative chat director 2024-10-09 21:23:18 -07:00
Debanjum Singh Solanky
6ad85e2275 Fix to continue showing retrieved documents in train of thought 2024-10-09 21:20:22 -07:00
sabaimran
a6f6e4f418 Fix notes references and passage of user query in the chat flow 2024-10-09 20:34:20 -07:00
Debanjum Singh Solanky
ec248efd31 Allow iterative chat director to do notes search 2024-10-09 19:04:59 -07:00
Debanjum Singh Solanky
a6905a9f0c Pass background context to iterating chat director 2024-10-09 19:04:59 -07:00
sabaimran
028b6e6379 Fix yield for scraping direct web page 2024-10-09 18:14:08 -07:00
sabaimran
717d9da8d8 Handle when summarize result is not present, rename variable in for loop from query 2024-10-09 17:57:08 -07:00
sabaimran
03544efde2 Ignore typing of the result dict for online, web page scrape 2024-10-09 17:48:24 -07:00
sabaimran
ab81b01fcb Fix typing of direct_web_pages and remove the deprecated chat API 2024-10-09 17:46:28 -07:00
sabaimran
5b8d663cf1 Add intermediate summarization of results when planning with o1 2024-10-09 17:40:56 -07:00
sabaimran
7b288a1179 Clean up the function planning prompt a little bit 2024-10-09 16:59:20 -07:00
sabaimran
f71e4969d3 Skip summarize while it's broken, and snip some other parts of the workflow while under construction 2024-10-09 16:40:06 -07:00
sabaimran
f7e6f99a32 add typing for extract document references 2024-10-09 16:05:34 -07:00
sabaimran
6960fb097c update types of prev iterations response 2024-10-09 16:04:39 -07:00
sabaimran
4978360852 Fix type of previous_iterations 2024-10-09 16:02:41 -07:00
sabaimran
46ef205a75 Add additional type annotations for compiled_references et al 2024-10-09 16:01:52 -07:00
sabaimran
4fbaef10e9 Correct usage of the summarize function 2024-10-09 15:58:05 -07:00
sabaimran
c91678078d Correct the usage of query passed to summarize function 2024-10-09 15:55:55 -07:00
sabaimran
f867d5ed72 Working prototype of meta-level chain of reasoning and execution 
- Create a more dynamic reasoning agent that can evaluate information and understand what it doesn't know, making moves to get that information
- Lots of hacks and code that needs to be reversed later on before submission
 2024-10-09 15:54:25 -07:00
425 changed files with 58849 additions and 27119 deletions
Expand all files Collapse all files

50
.devcontainer/Dockerfile Normal file
View File

@@ -0,0 +1,50 @@
ARG PYTHON_VERSION=3.12
FROM mcr.microsoft.com/devcontainers/python:${PYTHON_VERSION}
# Install UV and Bun
RUN curl -fsSL https://bun.sh/install | bash && mv /root/.bun/bin/bun /usr/local/bin/bun
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
RUN uv python pin $PYTHON_VERSION
# create python virtual environment
RUN uv venv /opt/venv --python $PYTHON_VERSION --seed
# Add venv to PATH for subsequent RUN commands and for the container environment
ENV PATH="/opt/venv/bin:$PATH"
# Tell pip, uv to use this virtual environment
ENV VIRTUAL_ENV="/opt/venv"
ENV UV_PROJECT_ENVIRONMENT="/opt/venv"
# Setup working directory
WORKDIR /workspaces/khoj
# --- Python Server App Dependencies ---
# Copy files required for Python dependency installation.
COPY pyproject.toml README.md ./
# Setup python environment
# Use the pre-built torch cpu wheel
ENV UV_INDEX="https://download.pytorch.org/whl/cpu" \
UV_INDEX_STRATEGY="unsafe-best-match" \
# Avoid downloading unused cuda specific python packages
CUDA_VISIBLE_DEVICES="" \
# Use static version to build app without git dependency
VERSION=0.0.0 \
# Use embedded db
USE_EMBEDDED_DB="True" \
PGSERVER_DATA_DIR="/opt/khoj_db"
# Install Python dependencies from pyproject.toml in editable mode
RUN sed -i "s/dynamic = \\[\"version\"\\]/version = \"$VERSION\"/" pyproject.toml && \
uv sync --all-extras && \
# Save the lock file generated with correct Linux platform wheels
cp uv.lock /opt/uv.lock.linux && \
chown -R vscode:vscode /opt/venv
# --- Web App Dependencies ---
# Copy web app manifest files
COPY src/interface/web/package.json src/interface/web/bun.lock /opt/khoj_web/
# Install web app dependencies
RUN cd /opt/khoj_web && bun install && chown -R vscode:vscode .
# The .venv and node_modules are now populated in the image.
# The rest of the source code will be mounted by VS Code from your local checkout,
# overlaying any files copied here if they are part of the workspace mount.

58
.devcontainer/devcontainer.json Normal file
View File

@@ -0,0 +1,58 @@
{
"name": "Khoj Development Environment",
"build": {
"dockerfile": "Dockerfile",
"context": "..", // Build context is the project root
"args": {
"PYTHON_VERSION": "3.12"
}
},
"forwardPorts": [
42110
],
"containerEnv": {
"USE_EMBEDDED_DB": "True"
},
"customizations": {
"vscode": {
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"ms-python.black-formatter",
"ms-python.mypy-type-checker",
"ms-python.isort",
"esbenp.prettier-vscode",
"GitHub.copilot",
"GitHub.copilot-chat",
"GitHub.vscode-pull-request-github",
"github.vscode-github-actions",
"unifiedjs.vscode-mdx"
],
"settings": {
"python.defaultInterpreterPath": "/opt/venv/bin/python",
"python.formatting.provider": "black",
"python.linting.enabled": true,
"python.linting.mypyEnabled": true,
"python.linting.mypyArgs": [
"--config-file=pyproject.toml"
],
"mypy.configFile": "pyproject.toml",
"isort.args": [
"--profile",
"black",
"--filter-files"
],
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true,
}
}
},
"postCreateCommand": "scripts/dev_setup.sh --devcontainer",
"features": {
"ghcr.io/devcontainers/features/github-cli:1": {},
},
"remoteUser": "vscode"
}

29
.devcontainer/launch.json Normal file
View File

@@ -0,0 +1,29 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Launch Khoj",
"type": "debugpy",
"request": "launch",
"module": "src.khoj.main",
"console": "integratedTerminal",
"justMyCode": false,
"sudo": true,
"args": [
"-v",
"--anonymous-mode",
"--non-interactive",
"--port=42110"
],
"envFile": "${workspaceFolder}/.env",
"env": {
"KHOJ_ADMIN_EMAIL": "admin",
"KHOJ_ADMIN_PASSWORD": "admin",
"USE_EMBEDDED_DB": "true",
"KHOJ_DEBUG": "true",
"KHOJ_TELEMETRY_DISABLED": "true",
"PROMPTRACE_DIR": "${workspaceFolder}/promptrace"
}
}
]
}

13
.dockerignore
View File

@@ -1,10 +1,11 @@
.git/
.pytest_cache/
.vscode/
.venv/
docs/
.*
**/__pycache__/
*.egg-info/
documentation/
tests/
build/
dist/
scripts/
*.egg-info/
src/interface/
src/telemetry/
!src/interface/web

42
.github/ISSUE_TEMPLATE/bug-report.md vendored
View File

@@ -1,42 +0,0 @@
---
name: Bug Report
about: Create a bug to help fix something that might not be working correctly
title: "[FIX]"
labels: fix
assignees: ''
---
## Describe the bug
A clear and concise description of what the bug is. Please include what you were expecting to happen vs. what actually happened.
## To Reproduce
Steps to reproduce the behavior:
## Screenshots
If applicable, add screenshots to help explain your problem.
## Platform
- Server:
- [ ] Cloud-Hosted (https://app.khoj.dev)
- [ ] Self-Hosted Docker
- [ ] Self-Hosted Python package
- [ ] Self-Hosted source code
- Client:
- [ ] Obsidian
- [ ] Emacs
- [ ] Desktop app
- [ ] Web browser
- [ ] WhatsApp
- OS:
- [ ] Windows
- [ ] macOS
- [ ] Linux
- [ ] Android
- [ ] iOS
### If self-hosted
- Server Version [e.g. 1.0.1]:
## Additional context
Add any other context about the problem here.

144
.github/ISSUE_TEMPLATE/bug-report.yml vendored Normal file
View File

@@ -0,0 +1,144 @@
name: "🪲 Bug Report"
description: "Use this template to report a bug."
labels: ["fix"]
body:
- type: "markdown"
attributes:
value: |
> [!IMPORTANT]
> To save time for both you and us, try follow these guidelines before submitting a new issue:
> 1. Check if there is an existing issue tracking your bug on our Github.
> 2. When unsure if your issue is an actual bug, first discuss it on a [Github discussion](https://github.com/khoj-ai/khoj/discussions/new?category=q-a) or the **#bugs** channel of our [Discord server](https://discord.gg/b6gUdpKr).
> These steps avoid opening issues which are duplicate or not actual bugs.
- type: "checkboxes"
id: "server"
attributes:
label: "Server"
description: "With which Khoj server are you experiencing the issue? (select at least one)"
options:
- label: "Cloud (https://app.khoj.dev)"
required: false
- label: "Self-Hosted Docker"
required: false
- label: "Self-Hosted Python package"
required: false
- label: "Self-Hosted source code"
required: false
validations:
required: true
- type: "checkboxes"
id: "clients"
attributes:
label: "Clients"
description: "With which Khoj client(s) are you experiencing the issue? (select at least one)"
options:
- label: "Web browser"
required: false
- label: "Desktop/mobile app"
required: false
- label: "Obsidian"
required: false
- label: "Emacs"
required: false
- label: "WhatsApp"
required: false
validations:
required: true
- type: "checkboxes"
id: "os"
attributes:
label: "OS"
description: "On which operating system do you experience the issue? (select at least one)"
options:
- label: "Windows"
required: false
- label: "macOS"
required: false
- label: "Linux"
required: false
- label: "Android"
required: false
- label: "iOS"
required: false
validations:
required: true
- type: "input"
id: "version"
attributes:
label: "Khoj version"
description: "Use `/help` command on the chat page to find the server version.\n
If using the cloud service, you can input, latest.\n
If self-hosting - please make sure to run the latest version of Khoj before reporting any issues as it may have already been fixed."
validations:
required: true
- type: "textarea"
id: "description"
attributes:
label: "Describe the bug"
description: "What is the problem? A clear and concise description of the bug."
validations:
required: true
- type: "textarea"
id: "current"
attributes:
label: "Current Behavior"
description: "What actually happened?\n\n
Please include full errors, uncaught exceptions, stack traces, screenshots and other relevant logs."
validations:
required: true
- type: "textarea"
id: "expected"
attributes:
label: "Expected Behavior"
description: "What did you expect to happen?"
validations:
required: true
- type: "textarea"
id: "reproduction"
attributes:
label: "Reproduction Steps"
description: "Detail the steps needed to reproduce the issue. This can include a self-contained, concise snippet of
code, if applicable.\n\n
For more complex issues, provide a link to a repository with the smallest sample that reproduces
the bug.\n
If the issue can be replicated without code, please provide a clear, step-by-step description of
the actions or conditions necessary to reproduce it. Any screenshots are also appreciated.\n
Avoid including business logic or unrelated details, as this makes diagnosis more difficult.\n\n
Whether it's a sequence of actions, code samples, or specific conditions, ensure that the steps
are clear enough to be easily followed and replicated."
validations:
required: true
- type: "textarea"
id: "workaround"
attributes:
label: "Possible Workaround"
description: "If you find any workaround for this problem - please, provide it here."
validations:
required: false
- type: "textarea"
id: "context"
attributes:
label: "Additional Information"
description: "Anything else that might be relevant for troubleshooting this bug.\n
Providing context helps us come up with a solution that is most useful in the real-world use case.\n
For example if self-hosting, provide environment details like OS versin, Docker version etc."
validations:
required: false
- type: "input"
id: "discussion_link"
attributes:
label: "Link to Discord or Github discussion"
description: "Provide a link to the first message of bug's discussion on Discord or Github.\n
This will help to keep history of why this bug exists."
validations:
required: false

11
.github/ISSUE_TEMPLATE/feature-request.md vendored
View File

@@ -1,11 +0,0 @@
---
name: Feature Request
about: Suggest an idea to help make Khoj a better tool
title: "[IDEA]"
labels: "upgrade"
assignees: ''
---
## Describe the feature you'd like
A clear and concise description of what you want to happen. Include any relevant links or screenshots or inspiration.

54
.github/ISSUE_TEMPLATE/feature-request.yml vendored Normal file
View File

@@ -0,0 +1,54 @@
name: "🙋 Feature Request"
description: "Use this template to request new feature or suggest an idea for Khoj"
labels: ["upgrade"]
body:
- type: "markdown"
attributes:
value: |
> [!IMPORTANT]
> To save time for both you and us, try follow these guidelines before submitting a feature request:
> 1. Check if there is an existing feature request that is similar to your on our Github.
> 2. We encourage you to first discuss your idea on a [Github discussion](https://github.com/khoj-ai/khoj/discussions/categories/ideas) or the **#ideas** channel of our [Discord server](https://discord.gg/b6gUdpKr).
> This step helps in understanding the new feature and determining if it's can be implemented at all.
Only proceed with this report if your idea was approved after the GitHub/Discord discussion.
- type: "textarea"
id: "description"
attributes:
label: "Describe the feature"
description: "A clear and concise description of the feature you are proposing."
validations:
required: true
- type: "textarea"
id: "use-case"
attributes:
label: "Use Case"
description: "Why do you need this feature? Provide real world use cases, the more the better."
validations:
required: true
- type: "textarea"
id: "solution"
attributes:
label: "Proposed Solution"
description: "Suggest how to implement the new feature. Please include prototype/sketch/reference implementation."
validations:
required: false
- type: "textarea"
id: "additional_info"
attributes:
label: "Additional Information"
description: "Any additional information you would like to provide - links, screenshots, etc."
validations:
required: false
- type: "input"
id: "discussion_link"
attributes:
label: "Link to Discord or Github discussion"
description: "Provide a link to the first message of feature request's discussion on Discord or Github.\n
This will help to keep history of why this feature request exists."
validations:
required: false

2
.github/workflows/build_khoj_el.yml vendored
View File

@@ -6,12 +6,14 @@ on:
push:
branches:
- 'master'
- 'release/1.x'
paths:
- src/interface/emacs/*.el
- .github/workflows/build_khoj_el.yml
pull_request:
branches:
- 'master'
- 'release/1.x'
paths:
- src/interface/emacs/*.el
- .github/workflows/build_khoj_el.yml

19
.github/workflows/desktop.yml vendored
View File

@@ -6,6 +6,7 @@ on:
- "*"
branches:
- 'master'
- 'release/1.x'
paths:
- src/interface/desktop/**
- .github/workflows/desktop.yml
@@ -42,10 +43,10 @@ jobs:
run: |
npx todesktop build
- name: 📦 Release Desktop App
if: startsWith(github.ref, 'refs/tags/')
run: |
npx todesktop release --latest --force
# - name: 📦 Release Desktop App
# if: startsWith(github.ref, 'refs/tags/')
# run: |
# npx todesktop release --latest --force
- name: ⤵️ Get Desktop Apps
if: startsWith(github.ref, 'refs/tags/')
@@ -60,7 +61,7 @@ jobs:
- name: ⏫ Upload Mac ARM App
if: startsWith(github.ref, 'refs/tags/')
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
if-no-files-found: warn
name: khoj-${{ github.ref_name }}-arm64.dmg
@@ -68,7 +69,7 @@ jobs:
- name: ⏫ Upload Mac x64 App
if: startsWith(github.ref, 'refs/tags/')
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
if-no-files-found: warn
name: khoj-${{ github.ref_name }}-x64.dmg
@@ -76,7 +77,7 @@ jobs:
- name: ⏫ Upload Windows App
if: startsWith(github.ref, 'refs/tags/')
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
if-no-files-found: warn
name: khoj-${{ github.ref_name }}-x64.exe
@@ -84,7 +85,7 @@ jobs:
- name: ⏫ Upload Debian App
if: startsWith(github.ref, 'refs/tags/')
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
if-no-files-found: warn
name: khoj-${{ github.ref_name }}-x64.deb
@@ -92,7 +93,7 @@ jobs:
- name: ⏫ Upload Linux App Image
if: startsWith(github.ref, 'refs/tags/')
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
if-no-files-found: warn
name: khoj-${{ github.ref_name }}-x64.AppImage

124
.github/workflows/dockerize.yml vendored
View File

@@ -6,12 +6,14 @@ on:
- "*"
branches:
- master
- release/1.x
paths:
- src/khoj/**
- src/interface/web/**
- pyproject.toml
- Dockerfile
- prod.Dockerfile
- computer.Dockerfile
- docker-compose.yml
- .github/workflows/dockerize.yml
workflow_dispatch:
@@ -27,24 +29,38 @@ on:
description: 'Build Khoj cloud docker image'
type: boolean
default: true
khoj-computer:
description: 'Build computer for Khoj'
type: boolean
default: true
env:
# Tag Image with tag name on release
# else with user specified tag (default 'dev') if triggered via workflow
# else with run_id if triggered via a pull request
# else with 'pre' (if push to master)
DOCKER_IMAGE_TAG: ${{ github.ref_type == 'tag' && github.ref_name || github.event_name == 'workflow_dispatch' && github.event.inputs.tag || 'pre' }}
# else with 'pre' (if push to master) or 'pre-1x' (if push to release/1.x)
DOCKER_IMAGE_TAG: ${{ github.ref_type == 'tag' && github.ref_name || github.event_name == 'workflow_dispatch' && github.event.inputs.tag || github.ref_name == 'release/1.x' && 'pre-1x' || 'pre' }}
jobs:
build:
name: Publish Khoj Docker Images
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
image:
- 'local'
- 'cloud'
include:
- image: 'local'
platform: linux/amd64
runner: ubuntu-latest
- image: 'local'
platform: linux/arm64
runner: ubuntu-linux-arm64
- image: 'cloud'
platform: linux/amd64
runner: ubuntu-latest
- image: 'cloud'
platform: linux/arm64
runner: ubuntu-linux-arm64
runs-on: ${{ matrix.runner }}
steps:
- name: Checkout Code
uses: actions/checkout@v3
@@ -52,9 +68,6 @@ jobs:
# Get all history to correctly infer Khoj version using hatch
fetch-depth: 0
- name: 🧹 Delete huge unnecessary tools folder
run: rm -rf /opt/hostedtoolcache
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
@@ -73,31 +86,108 @@ jobs:
run: rm -rf /opt/hostedtoolcache
- name: 📦 Build and Push Docker Image
uses: docker/build-push-action@v2
uses: docker/build-push-action@v4
if: (matrix.image == 'local' && github.event_name == 'workflow_dispatch') && github.event.inputs.khoj == 'true' || (matrix.image == 'local' && github.event_name == 'push')
with:
context: .
file: Dockerfile
platforms: linux/amd64, linux/arm64
push: true
tags: |
ghcr.io/${{ github.repository }}:${{ env.DOCKER_IMAGE_TAG }}
${{ github.ref_type == 'tag' && format('ghcr.io/{0}:latest', github.repository) || '' }}
ghcr.io/${{ github.repository }}:${{ env.DOCKER_IMAGE_TAG }}-${{ matrix.platform == 'linux/amd64' && 'amd64' || 'arm64' }}
build-args: |
VERSION=${{ steps.hatch.outputs.version }}
PORT=42110
cache-from: type=gha,scope=${{ matrix.image }}-${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=${{ matrix.image }}-${{ matrix.platform}}
labels: |
org.opencontainers.image.description=Khoj AI - Your second brain powered by LLMs and Neural Search
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
- name: 📦️⛅️ Build and Push Cloud Docker Image
uses: docker/build-push-action@v2
uses: docker/build-push-action@v4
if: (matrix.image == 'cloud' && github.event_name == 'workflow_dispatch') && github.event.inputs.khoj-cloud == 'true' || (matrix.image == 'cloud' && github.event_name == 'push')
with:
context: .
file: prod.Dockerfile
platforms: linux/amd64, linux/arm64
push: true
tags: |
ghcr.io/${{ github.repository }}-cloud:${{ env.DOCKER_IMAGE_TAG }}
${{ github.ref_type == 'tag' && format('ghcr.io/{0}-cloud:latest', github.repository) || '' }}
ghcr.io/${{ github.repository }}-cloud:${{ env.DOCKER_IMAGE_TAG }}-${{ matrix.platform == 'linux/amd64' && 'amd64' || 'arm64' }}
build-args: |
VERSION=${{ steps.hatch.outputs.version }}
PORT=42110
cache-from: type=gha,scope=${{ matrix.image }}-${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=${{ matrix.image }}-${{ matrix.platform}}
labels: |
org.opencontainers.image.description=Khoj AI Cloud - Your second brain powered by LLMs and Neural Search
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
- name: 📦️️💻 Build and Push Computer for Khoj
uses: docker/build-push-action@v4
if: github.event_name == 'workflow_dispatch' && github.event.inputs.khoj-computer == 'true' && matrix.image == 'local'
with:
context: .
file: computer.Dockerfile
push: true
tags: |
ghcr.io/${{ github.repository }}-computer:${{ env.DOCKER_IMAGE_TAG }}-${{ matrix.platform == 'linux/amd64' && 'amd64' || 'arm64' }}
cache-from: type=gha,scope=computer-${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=computer-${{ matrix.platform }}
labels: |
org.opencontainers.image.description=Khoj AI Computer - A computer for your second brain to operate
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
manifest:
needs: build
runs-on: ubuntu-latest
if: github.event_name != 'pull_request'
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.PAT }}
- name: Create and Push Local Manifest
if: github.event.inputs.khoj == 'true' || github.event_name == 'push'
run: |
# Only put "latest.*" tag on stable releases (i.e 1.x, 2.x+)
if [[ "${{ env.DOCKER_IMAGE_TAG }}" =~ ^[1-9]\.[0-9]+\.[0-9]+$ ]]; then
LATEST_TAG="latest"
else
LATEST_TAG="${{ env.DOCKER_IMAGE_TAG }}"
fi
docker buildx imagetools create \
-t ghcr.io/${{ github.repository }}:${{ env.DOCKER_IMAGE_TAG }} \
-t ghcr.io/${{ github.repository }}:${LATEST_TAG} \
ghcr.io/${{ github.repository }}:${{ env.DOCKER_IMAGE_TAG }}-amd64 \
ghcr.io/${{ github.repository }}:${{ env.DOCKER_IMAGE_TAG }}-arm64
- name: Create and Push Cloud Manifest
if: github.event.inputs.khoj-cloud == 'true' || github.event_name == 'push'
run: |
# Only put "latest.*" tag on stable releases (i.e 1.x, 2.x+)
if [[ "${{ env.DOCKER_IMAGE_TAG }}" =~ ^[1-9]\.[0-9]+\.[0-9]+$ ]]; then
LATEST_TAG="latest"
else
LATEST_TAG="${{ env.DOCKER_IMAGE_TAG }}"
fi
docker buildx imagetools create \
-t ghcr.io/${{ github.repository }}-cloud:${{ env.DOCKER_IMAGE_TAG }} \
-t ghcr.io/${{ github.repository }}-cloud:${LATEST_TAG} \
ghcr.io/${{ github.repository }}-cloud:${{ env.DOCKER_IMAGE_TAG }}-amd64 \
ghcr.io/${{ github.repository }}-cloud:${{ env.DOCKER_IMAGE_TAG }}-arm64
- name: Create and Push Computer Manifest
if: github.event.inputs.khoj-computer == 'true'
run: |
docker buildx imagetools create \
-t ghcr.io/${{ github.repository }}-computer:${{ env.DOCKER_IMAGE_TAG }} \
-t ghcr.io/${{ github.repository }}-computer:${{ github.ref_type == 'tag' && 'latest' || env.DOCKER_IMAGE_TAG }} \
ghcr.io/${{ github.repository }}-computer:${{ env.DOCKER_IMAGE_TAG }}-amd64 \
ghcr.io/${{ github.repository }}-computer:${{ env.DOCKER_IMAGE_TAG }}-arm64

14
.github/workflows/github_pages_deploy.yml vendored
View File

@@ -1,4 +1,4 @@
name: build and deploy github pages for documentation
name: deploy documentation
on:
push:
branches:
@@ -17,12 +17,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
# 👇 Build steps
- name: Set up Node.js
uses: actions/setup-node@v3
uses: actions/setup-node@v6
with:
node-version: 18.x
node-version: lts/*
cache: yarn
cache-dependency-path: documentation/yarn.lock
- name: Install dependencies
@@ -35,12 +35,12 @@ jobs:
yarn build
# 👆 Build steps
- name: Setup Pages
uses: actions/configure-pages@v3
uses: actions/configure-pages@v5
- name: Upload artifact
uses: actions/upload-pages-artifact@v2
uses: actions/upload-pages-artifact@v3
with:
# 👇 Specify build output path
path: documentation/build
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v2
uses: actions/deploy-pages@v4

22
.github/workflows/pre-commit.yml vendored
View File

@@ -2,6 +2,9 @@ name: pre-commit
on:
pull_request:
branches:
- master
- release/1.x
paths:
- src/**
- tests/**
@@ -12,6 +15,7 @@ on:
push:
branches:
- master
- release/1.x
paths:
- src/khoj/**
- tests/**
@@ -31,18 +35,24 @@ jobs:
with:
fetch-depth: 0
- name: Set up Python 3.11
uses: actions/setup-python@v4
- name: Install uv
uses: astral-sh/setup-uv@v4
with:
python-version: 3.11
version: "latest"
- name: Set up Python 3.11
run: uv python install 3.11
- name: ⏬️ Install Dependencies
run: |
sudo apt update && sudo apt install -y libegl1
python -m pip install --upgrade pip
- name: ⬇️ Install Application
run: pip install --no-cache-dir --upgrade .[dev]
env:
UV_INDEX: "https://download.pytorch.org/whl/cpu"
UV_INDEX_STRATEGY: "unsafe-best-match"
CUDA_VISIBLE_DEVICES: ""
run: uv sync --all-extras
- name: 🌡️ Validate Application
run: pre-commit run --hook-stage manual --all
run: uv run pre-commit run --hook-stage manual --all

38
.github/workflows/pypi.yml vendored
View File

@@ -6,6 +6,7 @@ on:
- "*"
branches:
- 'master'
- 'release/1.x'
paths:
- src/khoj/**
- src/interface/web/**
@@ -14,6 +15,7 @@ on:
pull_request:
branches:
- 'master'
- 'release/1.x'
paths:
- src/khoj/**
- src/interface/web/**
@@ -32,25 +34,28 @@ jobs:
with:
fetch-depth: 0
- name: Set up Python 3.11
uses: actions/setup-python@v4
- name: Install uv
uses: astral-sh/setup-uv@v4
with:
python-version: '3.11'
version: "latest"
- name: Set up Python 3.11
run: uv python install 3.11.12
- name: ⬇️ Install Server
run: python -m pip install --upgrade pip && pip install --upgrade .
run: uv sync --all-extras
- name: Install bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: ⬇️ Install Web Client
run: |
yarn install
yarn pypiciexport
bun install
bun pypiciexport
working-directory: src/interface/web
- name: 📂 Copy Generated Files
run: |
mkdir -p src/khoj/interface/compiled
cp -r /opt/hostedtoolcache/Python/3.11.10/x64/lib/python3.11/site-packages/khoj/interface/compiled/* src/khoj/interface/compiled/
- name: ⚙️ Build Python Package
run: |
# Setup Environment for Reproducible Builds
@@ -59,13 +64,13 @@ jobs:
rm -rf dist
# Build PyPI Package
pipx run build
uv build
- name: 🌡️ Validate Python Package
run: |
# Validate PyPi Package
pipx run check-wheel-contents dist/*.whl --ignore W004
pipx run twine check dist/*
uv tool run check-wheel-contents dist/*.whl --ignore W002,W004
uv tool run twine check dist/*
- name: ⏫ Upload Python Package Artifacts
uses: actions/upload-artifact@v4
@@ -74,7 +79,8 @@ jobs:
path: dist/khoj-*.whl
- name: 📦 Publish Python Package to PyPI
if: startsWith(github.ref, 'refs/tags') || github.ref == 'refs/heads/master'
uses: pypa/gh-action-pypi-publish@v1.8.14
if: startsWith(github.ref, 'refs/tags') || github.ref == 'refs/heads/master' || github.ref == 'refs/heads/release/1.x'
uses: pypa/gh-action-pypi-publish@release/v1.12
with:
skip-existing: true
print-hash: true

6
.github/workflows/release.yml vendored
View File

@@ -35,21 +35,21 @@ jobs:
yarn run build --if-present
- name: ⏫ Upload Obsidian Plugin main.js
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
if-no-files-found: error
name: main.js
path: src/interface/obsidian/main.js
- name: ⏫ Upload Obsidian Plugin manifest.json
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
if-no-files-found: error
name: manifest.json
path: src/interface/obsidian/manifest.json
- name: ⏫ Upload Obsidian Plugin styles.css
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
if-no-files-found: error
name: styles.css

234
.github/workflows/run_evals.yml vendored Normal file
View File

@@ -0,0 +1,234 @@
name: eval
on:
# Run on every release
push:
tags:
- "*"
# Allow manual triggers from GitHub UI
workflow_dispatch:
inputs:
khoj_mode:
description: 'Khoj Mode (general/default/research)'
required: true
default: 'default'
type: choice
options:
- general
- default
- research
dataset:
description: 'Dataset to evaluate (frames/simpleqa)'
required: true
default: 'frames'
type: choice
options:
- frames
- simpleqa
- gpqa
- math500
sample_size:
description: 'Number of samples to evaluate'
required: false
default: 200
type: number
sandbox:
description: 'Code sandbox to use'
required: false
default: 'terrarium'
type: choice
options:
- terrarium
- e2b
chat_model:
description: 'Chat model to use'
required: false
default: 'gemini-2.5-flash'
type: string
max_research_iterations:
description: 'Maximum number of iterations in research mode'
required: false
default: 5
type: number
openai_api_key:
description: 'OpenAI API key'
required: false
default: ''
type: string
openai_base_url:
description: 'Base URL of OpenAI compatible API'
required: false
default: 'https://api.openai.com/v1'
type: string
randomize:
description: 'Randomize the sample of questions'
required: false
default: 'true'
type: choice
options:
- 'false'
- 'true'
dataset_seed:
description: 'Seed to deterministically shuffle questions'
required: false
default: ''
type: string
jobs:
eval:
runs-on: ubuntu-latest
strategy:
matrix:
# Use input from manual trigger if available, else run all combinations
khoj_mode: ${{ github.event_name == 'workflow_dispatch' && fromJSON(format('["{0}"]', inputs.khoj_mode)) || fromJSON('["general", "default", "research"]') }}
dataset: ${{ github.event_name == 'workflow_dispatch' && fromJSON(format('["{0}"]', inputs.dataset)) || fromJSON('["frames", "gpqa"]') }}
services:
postgres:
image: ankane/pgvector
env:
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Install uv
uses: astral-sh/setup-uv@v4
with:
version: "latest"
- name: Set up Python
run: uv python install 3.10
- name: Get App Version
id: hatch
run: |
# Mask relevant workflow inputs as secret early
OPENAI_API_KEY=$(jq -r '.inputs.openai_api_key' $GITHUB_EVENT_PATH)
echo ::add-mask::$OPENAI_API_KEY
echo OPENAI_API_KEY="$OPENAI_API_KEY" >> $GITHUB_ENV
# Get app version from hatch
echo "version=$(pipx run hatch version)" >> $GITHUB_OUTPUT
- name: ⏬️ Install Dependencies
env:
DEBIAN_FRONTEND: noninteractive
run: |
# install dependencies
sudo apt update && sudo apt install -y git libegl1 sqlite3 libsqlite3-dev libsqlite3-0 ffmpeg libsm6 libxext6
# install terrarium for code sandbox
git clone https://github.com/khoj-ai/terrarium.git && cd terrarium && npm install --legacy-peer-deps && mkdir pyodide_cache
- name: ⬇️ Install Application
env:
UV_INDEX: "https://download.pytorch.org/whl/cpu"
UV_INDEX_STRATEGY: "unsafe-best-match"
CUDA_VISIBLE_DEVICES: ""
run: |
sed -i 's/dynamic = \["version"\]/version = "${{ steps.hatch.outputs.version }}"/' pyproject.toml
uv sync --all-extras
- name: 📝 Run Eval
env:
KHOJ_MODE: ${{ matrix.khoj_mode }}
SAMPLE_SIZE: ${{ github.event_name == 'workflow_dispatch' && inputs.sample_size || 200 }}
BATCH_SIZE: "20"
RANDOMIZE: ${{ github.event_name == 'workflow_dispatch' && inputs.randomize || 'true' }}
DATASET_SEED: ${{ github.event_name == 'workflow_dispatch' && inputs.dataset_seed || github.run_id }}
KHOJ_LLM_SEED: "42"
KHOJ_URL: "http://localhost:42110"
KHOJ_DEFAULT_CHAT_MODEL: ${{ github.event_name == 'workflow_dispatch' && inputs.chat_model || 'gemini-2.5-flash' }}
KHOJ_RESEARCH_ITERATIONS: ${{ github.event_name == 'workflow_dispatch' && inputs.max_research_iterations || 10 }}
KHOJ_AUTO_READ_WEBPAGE: ${{ github.event_name == 'workflow_dispatch' && inputs.auto_read_webpage || 'false' }}
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
OPENAI_BASE_URL: ${{ github.event_name == 'workflow_dispatch' && inputs.openai_base_url || 'https://api.openai.com/v1' }}
SERPER_DEV_API_KEY: ${{ matrix.dataset != 'math500' && secrets.SERPER_DEV_API_KEY || '' }}
OLOSTEP_API_KEY: ${{ matrix.dataset != 'math500' && secrets.OLOSTEP_API_KEY || ''}}
FIRECRAWL_API_KEY: ${{ matrix.dataset != 'math500' && secrets.FIRECRAWL_API_KEY || '' }}
HF_TOKEN: ${{ secrets.HF_TOKEN }}
E2B_API_KEY: ${{ inputs.sandbox == 'e2b' && secrets.E2B_API_KEY || '' }}
E2B_TEMPLATE: ${{ vars.E2B_TEMPLATE }}
KHOJ_ADMIN_EMAIL: khoj
KHOJ_ADMIN_PASSWORD: khoj
POSTGRES_HOST: localhost
POSTGRES_PORT: 5432
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
USE_EMBEDDED_DB: "true"
KHOJ_TELEMETRY_DISABLE: "True" # To disable telemetry for tests
run: |
set -euo pipefail
# Start Khoj server in background
# Capture stdout/stderr to a log for debugging if startup fails
uv run khoj --anonymous-mode --non-interactive -vv > khoj_server.log 2>&1 &
KHOJ_PID=$!
echo "Started Khoj (PID=$KHOJ_PID)"
# Start code sandbox
npm install -g pm2
NODE_ENV=production npm run ci --prefix terrarium
# Wait for server to be ready
timeout=120
while ! curl -s http://localhost:42110/api/health > /dev/null; do
# If process died, surface logs and fail fast
if ! kill -0 "$KHOJ_PID" 2>/dev/null; then
echo "Khoj process exited before becoming healthy. Logs:" >&2
sed -n '1,200p' khoj_server.log >&2 || true
exit 1
fi
if [ $timeout -le 0 ]; then
echo "Timed out waiting for Khoj server. Partial logs:" >&2
sed -n '1,200p' khoj_server.log >&2 || true
exit 1
fi
echo "Waiting for Khoj server... ($timeout s left)"
sleep 2
timeout=$((timeout-2))
done
echo "Khoj server is healthy"
# Run evals
uv run python tests/evals/eval.py -d ${{ matrix.dataset }}
- name: Upload Results
if: always() # Upload results even if tests fail
uses: actions/upload-artifact@v4
with:
name: eval-results-${{ steps.hatch.outputs.version }}-${{ matrix.khoj_mode }}-${{ matrix.dataset }}
path: |
*_evaluation_results_*.csv
*_evaluation_summary_*.txt
khoj_server.log
- name: Display Results
if: always()
run: |
# Read and display summary
echo "## Evaluation Summary of Khoj on ${{ matrix.dataset }} in ${{ matrix.khoj_mode }} mode" >> $GITHUB_STEP_SUMMARY
echo "**$(head -n 1 *_evaluation_summary_*.txt)**" >> $GITHUB_STEP_SUMMARY
echo "- Khoj Version: ${{ steps.hatch.outputs.version }}" >> $GITHUB_STEP_SUMMARY
echo "- Chat Model: ${{ inputs.chat_model || 'gemini-2.5-flash' }}" >> $GITHUB_STEP_SUMMARY
echo "- Code Sandbox: ${{ inputs.sandbox || 'terrarium' }}" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
tail -n +2 *_evaluation_summary_*.txt >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
# Display in logs too
echo "===== EVALUATION RESULTS ====="
cat *_evaluation_summary_*.txt

38
.github/workflows/test.yml vendored
View File

@@ -2,9 +2,13 @@ name: test
on:
pull_request:
branches:
- master
- release/1.x
paths:
- src/khoj/**
- tests/**
- '!tests/evals/**'
- config/**
- pyproject.toml
- .pre-commit-config.yml
@@ -12,9 +16,11 @@ on:
push:
branches:
- master
- release/1.x
paths:
- src/khoj/**
- tests/**
- '!tests/evals/**'
- config/**
- pyproject.toml
- .pre-commit-config.yml
@@ -24,7 +30,7 @@ jobs:
test:
name: Run Tests
runs-on: ubuntu-latest
container: ubuntu:jammy
container: ubuntu:latest
strategy:
fail-fast: false
matrix:
@@ -48,35 +54,32 @@ jobs:
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v4
- name: Install uv
uses: astral-sh/setup-uv@v4
with:
python-version: ${{ matrix.python_version }}
version: "latest"
- name: Install Git
run: |
apt update && apt install -y git
- name: Set up Python ${{ matrix.python_version }}
run: uv python install ${{ matrix.python_version }}
- name: ⏬️ Install Dependencies
env:
DEBIAN_FRONTEND: noninteractive
run: |
apt update && apt install -y libegl1 sqlite3 libsqlite3-dev libsqlite3-0 ffmpeg libsm6 libxext6
apt update && apt install -y git libegl1 sqlite3 libsqlite3-dev libsqlite3-0 ffmpeg libsm6 libxext6
- name: ⬇️ Install Postgres
env:
DEBIAN_FRONTEND: noninteractive
run : |
apt install -y postgresql postgresql-client && apt install -y postgresql-server-dev-14
- name: ⬇️ Install pip
run: |
apt install -y python3-pip
python -m ensurepip --upgrade
python -m pip install --upgrade pip
apt install -y postgresql postgresql-client && apt install -y postgresql-server-dev-16
- name: ⬇️ Install Application
run: sed -i 's/dynamic = \["version"\]/version = "0.0.0"/' pyproject.toml && pip install --upgrade .[dev]
env:
UV_INDEX: "https://download.pytorch.org/whl/cpu"
UV_INDEX_STRATEGY: "unsafe-best-match"
CUDA_VISIBLE_DEVICES: ""
run: sed -i 's/dynamic = \["version"\]/version = "0.0.0"/' pyproject.toml && uv sync --all-extras
- name: 🧪 Test Application
env:
@@ -85,5 +88,6 @@ jobs:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
run: pytest
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
run: uv run pytest
timeout-minutes: 10

7
.github/workflows/test_khoj_el.yml vendored
View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- 'master'
- 'release/1.x'
paths:
- src/interface/emacs/*.el
- src/interface/emacs/tests/*.el
@@ -11,6 +12,7 @@ on:
pull_request:
branches:
- 'master'
- 'release/1.x'
paths:
- src/interface/emacs/*.el
- src/interface/emacs/tests/*.el
@@ -23,10 +25,9 @@ jobs:
fail-fast: false
matrix:
emacs_version:
- 27.1
- 27.2
- 28.1
- 28.2
- 29.4
- 30.2
- snapshot
steps:
- uses: purcell/setup-emacs@master

14
.gitignore vendored
View File

@@ -3,6 +3,9 @@
*.pt
tests/data/models
tests/data/embeddings
tests/evals/**.txt
tests/evals/**.csv
pgserver_data/
# External app artifacts
__pycache__
@@ -12,6 +15,7 @@ __pycache__
.vscode
.env
.venv/*
.obsidian
todesktop.json
# Build artifacts
@@ -42,3 +46,13 @@ src/interface/obsidian/main.js
# obsidian
data.json
# Android
src/interface/android/.gradle
src/interface/android/app/build
src/interface/android/build
src/interface/android/*.aab
src/interface/android/*.apk
src/interface/android/*.apk.idsig
src/interface/android/*.keystore
src/interface/android/local.properties

20
.pre-commit-config.yaml
View File

@@ -1,8 +1,12 @@
repos:
- repo: https://github.com/psf/black
rev: 23.1.0
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.12.3
hooks:
- id: black
- id: ruff-check
args: [ --fix ]
files: \.py$
- id: ruff-format
files: \.py$
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
@@ -12,21 +16,15 @@ repos:
# Exclude elisp files to not clear page breaks
exclude: \.el$
- id: check-json
exclude: (devcontainer\.json|launch\.json|settings\.json)$
- id: check-toml
- id: check-yaml
- repo: https://github.com/pycqa/isort
rev: 5.12.0
hooks:
- id: isort
name: isort (python)
args: ["--profile", "black", "--filter-files"]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.0.0
hooks:
- id: mypy
stages: [push, manual]
stages: [pre-push, manual]
pass_filenames: false
args:
- --config-file=pyproject.toml

37
.vscode/launch.json vendored Normal file
View File

@@ -0,0 +1,37 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Launch Khoj",
"type": "debugpy",
"request": "launch",
"module": "src.khoj.main",
"console": "integratedTerminal",
"justMyCode": false,
"sudo": false,
"args": [
"-v",
"--anonymous-mode",
// "--non-interactive",
"--port=42110",
// "--host=example.com",
// "--sslcert=ssl.crt",
// "--sslkey=ssl.key",
],
"envFile": "${workspaceFolder}/.env",
"env": {
"KHOJ_ADMIN_EMAIL": "admin",
"KHOJ_ADMIN_PASSWORD": "admin",
"USE_EMBEDDED_DB": "true",
"KHOJ_DEBUG": "true",
"KHOJ_TELEMETRY_DISABLED": "true",
// Configure Code Sandbox
// "KHOJ_TERRARIUM_URL": "http://localhost:8080",
// Enable Promptracer to debug prompt flows
// "PROMPTRACE_DIR": "\${workspaceFolder}/promptrace",
// Enable Khoj Operator
// "KHOJ_OPERATOR_ENABLED": "True",
}
},
]
}

11
.vscode/settings.json vendored Normal file
View File

@@ -0,0 +1,11 @@
{
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true,
"python.analysis.include": [
"src/khoj/**/*"
],
"python.analysis.aiHoverSummaries": false,
}

52
Dockerfile
View File

@@ -1,5 +1,5 @@
# syntax=docker/dockerfile:1
FROM ubuntu:jammy
FROM ubuntu:jammy AS base
LABEL homepage="https://khoj.dev"
LABEL repository="https://github.com/khoj-ai/khoj"
LABEL org.opencontainers.image.source="https://github.com/khoj-ai/khoj"
@@ -13,36 +13,52 @@ RUN apt update -y && apt -y install \
# Required by RapidOCR
libgl1 \
libglx-mesa0 \
libglib2.0-0 && \
# Required by Next.js Web app
curl -sL https://deb.nodesource.com/setup_20.x | bash - && \
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \
apt update -y && apt -y --no-install-recommends install nodejs yarn && \
libglib2.0-0 \
docker.io \
# Required by llama-cpp-python pre-built wheels. See #1628
musl-dev && \
ln -s /usr/lib/x86_64-linux-musl/libc.so /lib/libc.musl-x86_64.so.1 && \
# Clean up
apt clean && rm -rf /var/lib/apt/lists/*
# Install Application
# Build Server
FROM base AS server-deps
WORKDIR /app
COPY pyproject.toml .
COPY README.md .
ARG VERSION=0.0.0
# use the pre-built llama-cpp-python, torch cpu wheel
ENV PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cpu https://abetlen.github.io/llama-cpp-python/whl/cpu"
# avoid downloading unused cuda specific python packages
ENV CUDA_VISIBLE_DEVICES=""
RUN sed -i "s/dynamic = \\[\"version\"\\]/version = \"$VERSION\"/" pyproject.toml && \
pip install --no-cache-dir .
# Copy Source Code
COPY . .
# Set the PYTHONPATH environment variable in order for it to find the Django app.
ENV PYTHONPATH=/app/src:$PYTHONPATH
# Go to the directory src/interface/web and export the built Next.js assets
# Build Web App
FROM oven/bun:1-alpine AS web-app
# Set build optimization env vars
ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1
WORKDIR /app/src/interface/web
RUN bash -c "yarn install --frozen-lockfile --verbose && yarn ciexport && yarn cache clean"
# Install dependencies first (cache layer)
COPY src/interface/web/package.json src/interface/web/bun.lock ./
RUN bun install --frozen-lockfile
# Copy source and build
COPY src/interface/web/. ./
RUN bun run build
# Merge the Server and Web App into a Single Image
FROM base
ENV PYTHONPATH=/app/src
WORKDIR /app
COPY --from=server-deps /usr/local/lib/python3.10/dist-packages /usr/local/lib/python3.10/dist-packages
COPY --from=web-app /app/src/interface/web/out ./src/khoj/interface/built
COPY . .
RUN cd src && python3 khoj/manage.py collectstatic --noinput
# Run the Application
# There are more arguments required for the application to run,
# but these should be passed in through the docker-compose.yml file.
ARG PORT
# but those should be passed in through the docker-compose.yml file.
ARG PORT=42110
EXPOSE ${PORT}
ENTRYPOINT ["python3", "src/khoj/main.py"]

45
README.md
View File

@@ -1,4 +1,4 @@
<p align="center"><img src="src/khoj/interface/web/assets/icons/khoj-logo-sideways-500.png" width="230" alt="Khoj Logo"></p>
<p align="center"><img src="https://assets.khoj.dev/khoj-logo-sideways-1200x540.png" width="230" alt="Khoj Logo"></p>
<div align="center">
@@ -27,15 +27,23 @@
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
[✍🏽 Blog](https://blog.khoj.dev)
</div>
<a href="https://trendshift.io/repositories/10318" target="_blank"><img src="https://trendshift.io/api/badge/repositories/10318" alt="khoj-ai%2Fkhoj | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
<div align="left">
</div>
***
### 🎁 New
* Meet 🌶️ **[Pipali](https://pipali.ai)** - our [open-source](https://github.com/khoj-ai/pipali) AI coworker that runs on your computer.
* [Read](https://blog.khoj.dev/posts/evaluate-khoj-quality/) about Khoj's excellent performance on modern retrieval and reasoning benchmarks.
***
## Overview
[Khoj](https://khoj.dev) is a personal AI app to extend your capabilities. It smoothly scales up from an on-device personal AI to a cloud-scale enterprise AI.
- Chat with any local or online LLM (e.g llama3, qwen, gemma, mistral, gpt, claude, gemini).
- Chat with any local or online LLM (e.g llama3, qwen, gemma, mistral, gpt, claude, gemini, deepseek).
- Get answers from the internet and your docs (including image, pdf, markdown, org-mode, word, notion files).
- Access it from your Browser, Obsidian, Emacs, Desktop, Phone or Whatsapp.
- Create agents with custom knowledge, persona, chat model and tools to take on any role.
@@ -47,8 +55,6 @@
***
</div>
## See it in action
![demo_chat](https://github.com/khoj-ai/khoj/blob/master/documentation/assets/img/quadratic_equation_khoj_web.gif?raw=true)
@@ -62,6 +68,26 @@ You can see the full feature list [here](https://docs.khoj.dev/category/features
To get started with self-hosting Khoj, [read the docs](https://docs.khoj.dev/get-started/setup).
## Enterprise
Khoj is available as a cloud service, on-premises, or as a hybrid solution. To learn more about Khoj Enterprise, [visit our website](https://khoj.dev/teams).
## Frequently Asked Questions (FAQ)
Q: Can I use Khoj without self-hosting?
Yes! You can use Khoj right away at [https://app.khoj.dev](https://app.khoj.dev) — no setup required.
Q: What kinds of documents can Khoj read?
Khoj supports a wide variety: PDFs, Markdown, Notion, Word docs, org-mode files, and more.
Q: How can I make my own agent?
Check out [this blog post](https://blog.khoj.dev/posts/create-agents-on-khoj/) for a step-by-step guide to custom agents.
For more questions, head over to our [Discord](https://discord.gg/BDgyabRM6e)!
## Contributors
Cheers to our awesome contributors! 🎉
@@ -72,5 +98,10 @@ Cheers to our awesome contributors! 🎉
Made with [contrib.rocks](https://contrib.rocks).
### Interested in Contributing?
Khoj is open source. It is sustained by the community and we’d love for you to join it! Whether you’re a coder, designer, writer, or enthusiast, there’s a place for you.
We are always looking for contributors to help us build new features, improve the project documentation, or fix bugs. If you're interested, please see our [Contributing Guidelines](https://docs.khoj.dev/contributing/development) and check out our [Contributors Project Board](https://github.com/orgs/khoj-ai/projects/4).
Why Contribute?
- Make an Impact: Help build, test and improve a tool used by thousands to boost productivity.
- Learn & Grow: Work on cutting-edge AI, LLMs, and semantic search technologies.
You can help us build new features, improve the project documentation, report issues and fix bugs. If you're a developer, please see our [Contributing Guidelines](https://docs.khoj.dev/contributing/development) and check out [good first issues](https://github.com/khoj-ai/khoj/contribute) to work on.

130
computer.Dockerfile Normal file
View File

@@ -0,0 +1,130 @@
FROM ubuntu:24.04
ENV DEBIAN_FRONTEND=noninteractive
# Install System Dependencies
RUN apt update \
&& apt install -y \
ca-certificates \
gnupg \
xfce4 \
xfce4-goodies \
x11vnc \
xvfb \
xdotool \
imagemagick \
x11-apps \
dbus-x11 \
sudo \
python3-pip \
python3-tk \
python3-dev \
build-essential \
scrot \
gnome-screenshot \
net-tools \
libx11-dev \
libxext-dev \
libxtst-dev \
libxinerama-dev \
libxmu-dev \
libxrandr-dev \
libxfixes-dev \
software-properties-common \
&& add-apt-repository ppa:mozillateam/ppa && apt update \
&& apt install -y --no-install-recommends \
# Desktop apps
firefox-esr \
libreoffice \
x11-apps \
xpdf \
gedit \
xpaint \
tint2 \
galculator \
pcmanfm \
unzip \
# Terminal apps like file editors, viewers, git, wget/curl etc.
less \
nano \
neovim \
vim \
git \
curl \
wget \
procps \
# Python/pyenv dependencies
libssl-dev \
zlib1g-dev \
libbz2-dev \
libreadline-dev \
libsqlite3-dev \
libncursesw5-dev \
xz-utils \
tk-dev \
libxml2-dev \
libxmlsec1-dev \
libffi-dev \
liblzma-dev \
# set default browser
&& update-alternatives --set x-www-browser /usr/bin/firefox-esr \
&& apt-get clean && rm -rf /var/lib/apt/lists/* \
# remove screen locks, power managers
&& apt remove -y light-locker xfce4-screensaver xfce4-power-manager || true
# Create Computer User
ENV USERNAME=khoj
ENV HOME=/home/$USERNAME
RUN groupadd $USERNAME && \
useradd -m -s /bin/bash -d $HOME -g $USERNAME $USERNAME && \
echo "${USERNAME} ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
USER $USERNAME
WORKDIR $HOME
# Install Python using uv and create a virtual environment
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
ENV PYTHON_VERSION=3.11.6
RUN uv python pin $PYTHON_VERSION
RUN uv venv $HOME/.venv --python $PYTHON_VERSION --seed
RUN echo 'export PATH="$HOME/.venv/bin:$PATH"' >> "$HOME/.bashrc"
ENV PATH="$HOME/.venv/bin:$PATH"
# Install Python Packages
RUN uv pip install --no-cache-dir \
pyautogui \
Pillow \
pyperclip \
pygetwindow
# Setup VNC
RUN x11vnc -storepasswd secret /home/khoj/.vncpass
ARG WIDTH=1024
ARG HEIGHT=768
ARG DISPLAY_NUM=99
ENV WIDTH=$WIDTH
ENV HEIGHT=$HEIGHT
ENV DISPLAY_NUM=$DISPLAY_NUM
ENV DISPLAY=":$DISPLAY_NUM"
# Expose VNC on port 5900
EXPOSE 5900
# Start Virtual Display (Xvfb), Desktop Manager (XFCE) and Remote Viewer (X11 VNC)
CMD ["/bin/sh", "-c", " \
# Create and permission XDG_RUNTIME_DIR with sudo \n\
export XDG_RUNTIME_DIR=/run/user/$(id -u); \
sudo mkdir -p $XDG_RUNTIME_DIR && \
sudo chown $(id -u):$(id -g) $XDG_RUNTIME_DIR && \
sudo chmod 0700 $XDG_RUNTIME_DIR; \
\
# Start Virtual Display \n\
Xvfb $DISPLAY -screen 0 ${WIDTH}x${HEIGHT}x24 -dpi 96 -auth /home/$USERNAME/.Xauthority >/dev/null 2>&1 & \
sleep 1; \
xauth add $DISPLAY . $(mcookie); \
\
# Start VNC Server \n\
x11vnc -display $DISPLAY -forever -rfbauth /home/$USERNAME/.vncpass -listen 0.0.0.0 -rfbport 5900 >/dev/null 2>&1 & \
eval $(dbus-launch --sh-syntax) && \
startxfce4 & \
sleep 2 && echo 'Container running!' && \
tail -f /dev/null "]

82
docker-compose.yml
View File

@@ -1,8 +1,6 @@
services:
database:
image: ankane/pgvector
ports:
- "5432:5432"
image: docker.io/pgvector/pgvector:pg15
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
@@ -14,13 +12,38 @@ services:
interval: 30s
timeout: 10s
retries: 5
sandbox:
image: ghcr.io/khoj-ai/terrarium:latest
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 2
search:
image: docker.io/searxng/searxng:latest
volumes:
- khoj_search:/etc/searxng
environment:
- SEARXNG_BASE_URL=http://localhost:8080/
# Creates Computer for Khoj to use.
# Set KHOJ_OPERATOR_ENABLED=True in the server service environment variable to enable.
computer:
container_name: khoj-computer
image: ghcr.io/khoj-ai/khoj-computer:latest
# build:
# context: .
# dockerfile: computer.Dockerfile
ports:
- "5900:5900"
volumes:
- khoj_computer:/home/operator
server:
depends_on:
database:
condition: service_healthy
# Use the following line to use the latest version of khoj. Otherwise, it will build from source.
# Use the following line to use the latest version of khoj. Otherwise, it will build from source. Set this to ghcr.io/khoj-ai/khoj-cloud:latest if you want to use the prod image.
image: ghcr.io/khoj-ai/khoj:latest
# Uncomment the following line to build from source. This will take a few minutes. Comment the next two lines out if you want to use the offiicial image.
# Uncomment the following line to build from source. This will take a few minutes. Comment the next two lines out if you want to use the official image.
# build:
# context: .
ports:
@@ -29,10 +52,15 @@ services:
# change the port in the args in the build section,
# as well as the port in the command section to match
- "42110:42110"
extra_hosts:
- "host.docker.internal:host-gateway"
working_dir: /app
volumes:
- khoj_config:/root/.khoj/
- khoj_models:/root/.cache/torch/sentence_transformers
- khoj_models:/root/.cache/huggingface
# uncomment line below to mount docker socket to allow khoj to use its computer.
# - /var/run/docker.sock:/var/run/docker.sock
# Use 0.0.0.0 to explicitly set the host ip for the service on the container. https://pythonspeed.com/articles/docker-connection-refused/
environment:
- POSTGRES_DB=postgres
@@ -44,23 +72,61 @@ services:
- KHOJ_DEBUG=False
- KHOJ_ADMIN_EMAIL=username@example.com
- KHOJ_ADMIN_PASSWORD=password
# Uncomment lines below to use chat models by each provider.
# Default URL of Terrarium, the default Python sandbox used by Khoj to run code. Its container is specified above
- KHOJ_TERRARIUM_URL=http://sandbox:8080
# Uncomment line below to have Khoj run code in remote E2B code sandbox instead of the self-hosted Terrarium sandbox above. Get your E2B API key from https://e2b.dev/.
# - E2B_API_KEY=your_e2b_api_key
# Default URL of SearxNG, the default web search engine used by Khoj. Its container is specified above
- KHOJ_SEARXNG_URL=http://search:8080
# Uncomment line below to use with Ollama running on your local machine at localhost:11434.
# Change URL to use with other OpenAI API compatible providers like VLLM, LMStudio, DeepInfra, DeepSeek etc.
# - OPENAI_BASE_URL=http://host.docker.internal:11434/v1/
# - KHOJ_DEFAULT_CHAT_MODEL=qwen3
#
# Uncomment appropriate lines below to use chat models by OpenAI, Anthropic, Google.
# Ensure you set your provider specific API keys.
# ---
# - OPENAI_API_KEY=your_openai_api_key
# - GEMINI_API_KEY=your_gemini_api_key
# - ANTHROPIC_API_KEY=your_anthropic_api_key
#
# Uncomment line below to enable Khoj to use its computer.
# - KHOJ_OPERATOR_ENABLED=True
# Uncomment appropriate lines below to enable web results with Khoj
# Ensure you set your provider specific API keys.
# ---
# Paid, Fast API. Only does web search. Get API key from https://serper.dev/
# - SERPER_DEV_API_KEY=your_serper_dev_api_key
# Paid, Higher Read Success API. Only does webpage read. Get API key from https://olostep.com/
# - OLOSTEP_API_KEY=your_olostep_api_key
# Paid, Open API. Does both web search and webpage read. Get API key from https://firecrawl.dev/
# - FIRECRAWL_API_KEY=your_firecrawl_api_key
# Paid, Fast API. Does both web search and webpage read. Get API key from https://exa.ai/
# - EXA_API_KEY=your_exa_api_key
#
# Uncomment the necessary lines below to make your instance publicly accessible.
# Replace the KHOJ_DOMAIN with either your domain or IP address (no http/https prefix).
# Proceed with caution, especially if you are using anonymous mode.
# ---
# - KHOJ_NO_HTTPS=True
# Replace the KHOJ_DOMAIN with the server's externally accessible domain or I.P address from a remote machie (no http/https prefix).
# Ensure this is set correctly to avoid CSRF trusted origin or unset cookie issue when trying to access the admin panel.
# - KHOJ_DOMAIN=192.168.0.104
# - KHOJ_DOMAIN=khoj.example.com
# Replace the KHOJ_ALLOWED_DOMAIN with the server's internally accessible domain or I.P address on the host machine (no http/https prefix).
# Only set if using a load balancer/reverse_proxy in front of your Khoj server. If unset, it defaults to KHOJ_DOMAIN.
# For example, if the load balancer service is added to the khoj docker network, set KHOJ_ALLOWED_DOMAIN to khoj's docker service name: `server'.
# - KHOJ_ALLOWED_DOMAIN=server
# - KHOJ_ALLOWED_DOMAIN=127.0.0.1
# Uncomment the line below to disable telemetry.
# Telemetry helps us prioritize feature development and understand how people are using Khoj
# Read more at https://docs.khoj.dev/miscellaneous/telemetry
# - KHOJ_TELEMETRY_DISABLE=True
# Comment out this line when you're using the official ghcr.io/khoj-ai/khoj-cloud:latest prod image.
command: --host="0.0.0.0" --port=42110 -vv --anonymous-mode --non-interactive
volumes:
khoj_config:
khoj_db:
khoj_models:
khoj_search:
khoj_computer:

BIN
documentation/assets/img/chrome_pwa_alt.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 500 KiB

BIN
documentation/assets/img/khoj_chat_on_desktop.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 298 KiB

BIN
documentation/assets/img/khoj_documentation.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 868 KiB

BIN
documentation/assets/img/khoj_search_on_desktop.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 333 KiB

BIN
documentation/assets/img/magic_link.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 71 KiB

After

Width:  |  Height:  |  Size: 23 KiB

BIN
documentation/assets/img/mic_chat_icon.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 8.4 KiB

BIN
documentation/assets/img/pwa_install_desktop.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

28
documentation/docs/advanced/admin.md
View File

@@ -1,13 +1,15 @@
# Admin Panel
> Describes the Khoj settings configurable via the admin panel
By default, your admin panel is available at `http://localhost:42110/server/admin/`. You can access the admin panel by logging in with your admin credentials (this would be your `KHOJ_ADMIN_EMAIL` and `KHOJ_ADMIN_PASSWORD`). The admin panel allows you to configure various settings for your Khoj server.
## App Settings
### Agents
Add all the agents you want to use for your different use-cases like Writer, Researcher, Therapist etc.
- `Personality`: This is a prompt to tell the chat model how to tune the personality of the agent.
- `Chat model`: The chat model to use for the agent.
- `Name`: The name of the agent. This field helps give the agent a unique identity across the app.
- `Avatar`: Url to the agents profile picture. It help give the agent a unique visual identity across the app.
- `Avatar`: Url to the agents profile picture. It helps give the agent a unique visual identity across the app.
- `Style color`, `Style icon`: These fields help give the agent a unique, visually identifiable identity across the app.
- `Slug`: This is the agent name to use in urls.
- `Public`: Check this if the agent is expected to be visible to all users on this Khoj server.
@@ -18,7 +20,7 @@ Add all the agents you want to use for your different use-cases like Writer, Res
### Chat Model Options
Add all the chat models you want to try, use and switch between for your different use-cases. For each chat model you add:
- `Chat model`: The name of an [OpenAI](https://platform.openai.com/docs/models), [Anthropic](https://docs.anthropic.com/en/docs/about-claude/models#model-names), [Gemini](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models#gemini-models) or [Offline](https://huggingface.co/models?pipeline_tag=text-generation&library=gguf) chat model.
- `Model type`: The chat model provider like `OpenAI`, `Offline`.
- `Model type`: The chat model provider like `OpenAI`, `Google`.
- `Vision enabled`: Set to `true` if your model supports vision. This is currently only supported for vision capable OpenAI models like `gpt-4o`
- `Max prompt size`, `Subscribed max prompt size`: These are optional fields. They are used to truncate the context to the maximum context size that can be passed to the model. This can help with accuracy and cost-saving.<br />
- `Tokenizer`: This is an optional field. It is used to accurately count tokens and truncate context passed to the chat model to stay within the models max prompt size.
@@ -36,25 +38,24 @@ To add a server chat setting:
- The `Advanced` field doesn't need to be set when self-hosting. When unset, the `Default` chat model is used for all users and the intermediate steps.
### OpenAI Processor Conversation Configs
These settings configure chat model providers to be accessed over API.
The name of this setting is kind of a misnomer, we know, it'll hopefully be changed at some point.
For each chat model provider you [add](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add):
### AI Model API
These settings configure APIs to interact with AI models.
For each AI Model API you [add](http://localhost:42110/server/admin/database/aimodelapi/add):
- `Api key`: Set to your [OpenAI](https://platform.openai.com/api-keys), [Anthropic](https://console.anthropic.com/account/keys) or [Gemini](https://aistudio.google.com/app/apikey) API keys.
- `Name`: Give the configuration any friendly name like `OpenAI`, `Gemini`, `Anthropic`.
- `Api base url`: Set the API base URL. This is only relevant to set if you're using another OpenAI-compatible proxy server like [Ollama](/advanced/ollama) or [LMStudio](/advanced/lmstudio).
![example configuration for openai processor](/img/example_openai_processor_config.png)
![example configuration for ai model api](/img/example_openai_processor_config.png)
### Search Model Configs
Search models are used to generate vector embeddings of your documents for natural language search and chat. You can choose any [embeddings models on HuggingFace](https://huggingface.co/models?pipeline_tag=sentence-similarity) to try, use for your to create vector embeddings of your documents for natural language search and chat.
Search models are used to generate vector embeddings of your documents for natural language search and chat. You can choose any [embeddings models on HuggingFace](https://huggingface.co/models?pipeline_tag=sentence-similarity) to create vector embeddings of your documents for natural language search and chat.
<img src="/img/example_search_model_admin_settings.png" alt="Example Search Model Settings" style={{width: 500}} />
### Text to Image Model Options
Add text to image generation models with these settings. Khoj currently supports text to image models available via OpenAI, Stability or Replicate API
- `api-key`: Set to your OpenAI, Stability or Replicate API key
Add text to image generation models with these settings. Khoj currently supports text to image models available via OpenAI, Google or Replicate API
- `api-key`: Set to your OpenAI, Google AI or Replicate API key
- `model`: Set the model name available over the selected model provider
- `model-type`: Set to the appropiate model provider
- `model-type`: Set to the appropriate model provider
- `openai-config`: For image generation models available via OpenAI (compatible) API you can set the appropriate OpenAI Processor Conversation Settings instead of specifying the `api-key` field above
### Speech to Text Model Options
@@ -63,6 +64,9 @@ Add speech to text models with these settings. Khoj currently only supports whis
### Voice Model Options
Add text to speech models with these settings. Khoj currently supports models from [ElevenLabs](https://elevenlabs.io/).
### Reflective Questions
This is a static list of starter question suggestions for each user. It is not currently used in any client app. It used to be shown on the web app home page. We may turn it into a dynamic list of starter questions personalized to each users, say based on their recent conversations or synced knowledge base.
## User Data
- Users, Entrys, Conversations, Subscriptions, Github configs, Notion configs, User search configs, User conversation configs, User voice configs
@@ -70,4 +74,4 @@ Add text to speech models with these settings. Khoj currently supports models fr
- Process Locks: Persistent Locks for Automations
- Client Applications:
Client applications allow you to setup third party applications that can query your Khoj server using a client application ID + secret. The secret would go in a bearer token.
Client applications allow you to setup third party applications that can query your Khoj server using a client application ID + secret. The secret would go in a bearer token.

52
documentation/docs/advanced/authentication.md
View File

@@ -1,52 +0,0 @@
# Authenticate
:::info
This is only helpful for self-hosted users or teams. If you're using [Khoj Cloud](https://app.khoj.dev), both Magic Links and Google OAuth work.
:::
By default, most of the instructions for self-hosting Khoj assume a single user, and so the default configuration is to run in anonymous mode. However, if you want to enable authentication, you can do so either with with [Magic Links](#using-magic-links) or [Google OAuth](#using-google-oauth) as shown below. This can be helpful to make Khoj securely accessible to you and your team.
:::tip[Note]
Remove the `--anonymous-mode` flag from your khoj start up command or docker-compose file to enable authentication.
:::
## Using Magic Links
The most secure way to do this is to integrate with [Resend](https://resend.com) by setting up an account and adding an environment variable for `RESEND_API_KEY`. You can get your API key [here](https://resend.com/api-keys). This will allow you to automatically send sign-in links to users who want to log in.
It's still possible to use the magic links feature without Resend, but you'll need to manually send the magic links to users who want to log in.
## Manually sending magic links
1. The user will have to enter their email address in the login form.
They'll click `Send Magic Link`. Without the Resend API key, this will just create an unverified account for them in the backend
<img src="/img/magic_link.png" alt="Magic link login form" width="400"/>
2. You can get their magic link using the admin panel
Go to the [admin panel](http://localhost:42110/server/admin/database/khojuser/). You'll see a list of users. Search for the user you want to send a magic link to. Tick the checkbox next to their row, and use the action drop down at the top to 'Get email login URL'. This will generate a magic link that you can send to the user, which will appear at the top of the admin interface.
| Get email login URL | Retrieved login URL |
|---------------------|---------------------|
| <img src="/img/admin_get_emali_login.png" alt="Get user magic sign in link" width="400" />| <img src="/img/admin_successful_login_url.png" alt="Successfully retrieved a login URL" width="400" />|
3. Send the magic link to the user. They can click on it to log in.
Once they click on the link, they'll automatically be logged in. They'll have to repeat this process for every new device they want to log in from, but they shouldn't have to repeat it on the same device.
A given magic link can only be used once. If the user tries to use it again, they'll be redirected to the login page to get a new magic link.
## Using Google OAuth
To set up your self-hosted Khoj with Google Auth, you need to create a project in the Google Cloud Console and enable the Google Auth API.
To implement this, you'll need to:
1. You must use the `python` package or build from source, because you'll need to install additional packages for the google auth libraries (`prod`). The syntax to install the right packages is
```
pip install khoj[prod]
```
2. [Create authorization credentials](https://developers.google.com/identity/sign-in/web/sign-in) for your application.
3. Open your [Google cloud console](https://console.developers.google.com/apis/credentials) and create a configuration like below for the relevant `OAuth 2.0 Client IDs` project:
![Google auth login project settings](https://github.com/khoj-ai/khoj/assets/65192171/9bcbf6f4-197d-4d0c-973a-c10b1331c892)
4. Configure these environment variables: `GOOGLE_CLIENT_SECRET`, and `GOOGLE_CLIENT_ID`. You can find these values in the Google cloud console, in the same place where you configured the authorized origins and redirect URIs.
That's it! That should be all you have to do. Now, when you reload Khoj without `--anonymous-mode`, you should be able to use your Google account to sign in.

74
documentation/docs/advanced/authentication.mdx Normal file
View File

@@ -0,0 +1,74 @@
# Authenticate (Multi-User Setup)
```mdx-code-block
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
```
By default, most of the instructions for self-hosting Khoj assume a single user, and so the default configuration is to run in anonymous mode. However, if you want to enable authentication, you can do so either with with [Magic Links](#using-magic-links) or [Google OAuth](#using-google-oauth) as shown below. This can be helpful to make Khoj securely accessible to you and your team.
:::tip[Note]
Remove the `--anonymous-mode` flag from your khoj start up command or docker-compose file to enable authentication.
:::
## Using Magic Links
The most secure way to do this is to integrate with [Resend](https://resend.com).
1. Setup your account at https://resend.com
2. Set an environment variable for `RESEND_API_KEY`. You can get your API key [here](https://resend.com/api-keys).
3. Set an environment variable for `RESEND_EMAIL`. This is the email address that will show up in your `from` field when sending magic links.
This will allow you to automatically send sign-in links to users who want to log in.
It's still possible to use the magic links feature without Resend, but you'll need to manually send the magic links to users who want to log in.
## Manually sending magic links
1. The user will have to enter their email address in the login popup shown at http://localhost:42110/?v=app.
They'll click `Get Login Link`. Without the Resend API key, this will just create an unverified account for them in the backend
<img src="/img/magic_link.png" alt="Magic link login form" width="400"/>
2. You can get their magic link using the admin panel
Go to the [admin panel](http://localhost:42110/server/admin/database/khojuser/). You'll see a list of users. Search for the user you want to send a magic link to. Tick the checkbox next to their row, and use the action drop down at the top to 'Get email login URL'. This will generate a magic link that you can send to the user, which will appear at the top of the admin interface.
| Get email login URL | Retrieved login URL |
|---------------------|---------------------|
| <img src="/img/admin_get_emali_login.png" alt="Get user magic sign in link" width="400" />| <img src="/img/admin_successful_login_url.png" alt="Successfully retrieved a login URL" width="400" />|
3. Send the magic link to the user. They can click on it to log in.
Once they click on the link, they'll automatically be logged in. They'll have to repeat this process for every new device they want to log in from, but they shouldn't have to repeat it on the same device.
A given magic link can only be used once. If the user tries to use it again, they'll be redirected to the login page to get a new magic link.
## Using Google OAuth
For this method, you'll need to use the prod version of the Khoj package. You can install it as below:
<Tabs groupId="server" queryString>
<TabItem value="docker" label="Docker">
Update your `docker-compose.yml` to use the prod image
```bash
image: ghcr.io/khoj-ai/khoj-cloud:latest
```
</TabItem>
<TabItem value="pip" label="Pip">
```bash
pip install khoj[prod]
```
</TabItem>
</Tabs>
To set up your self-hosted Khoj with Google Auth, you need to create a project in the Google Cloud Console and enable the Google Auth API.
To implement this, you'll need to:
1. [Create authorization credentials](https://developers.google.com/identity/sign-in/web/sign-in) for your application.
2. Open your [Google cloud console](https://console.developers.google.com/apis/credentials) and create a configuration like below for the relevant `OAuth 2.0 Client IDs` project:
![Google auth login project settings](https://github.com/khoj-ai/khoj/assets/65192171/9bcbf6f4-197d-4d0c-973a-c10b1331c892)
3. Configure these environment variables: `GOOGLE_CLIENT_SECRET`, and `GOOGLE_CLIENT_ID`. You can find these values in the Google cloud console, in the same place where you configured the authorized origins and redirect URIs.
That's it! That should be all you have to do. Now, when you reload Khoj without `--anonymous-mode`, you should be able to use your Google account to sign in.

45
documentation/docs/advanced/gcp-vertex.md Normal file
View File

@@ -0,0 +1,45 @@
# Google Vertex AI
:::info
This is only helpful for self-hosted users. If you're using [Khoj Cloud](https://app.khoj.dev), you can directly use any of the pre-configured AI models.
:::
Khoj can use Google's Gemini and Anthropic's Claude family of AI models from [Vertex AI](https://cloud.google.com/vertex-ai) on Google Cloud. Explore Anthropic and Gemini AI models available on Vertex AI's [Model Garden](https://console.cloud.google.com/vertex-ai/model-garden).
## Setup
1. Follow [these instructions](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#before_you_begin) to use models on GCP Vertex AI.
2. Create [Service Account](https://console.cloud.google.com/apis/credentials/serviceaccountkey) credentials.
- Download the credentials keyfile in json format.
- Base64 encode the credentials json keyfile. For example by running the following command from your terminal:
`base64 -i <service_account_credentials_keyfile.json>`
3. Create a new [API Model API](http://localhost:42110/server/admin/database/aimodelapi/add) on your Khoj admin panel.
- **Name**: `Google Vertex` (or whatever friendly name you prefer).
- **Api Key**: `base64 encoded json keyfile` from step 2.
- **Api Base Url**: `https://{MODEL_GCP_REGION}-aiplatform.googleapis.com/v1/projects/{YOUR_GCP_PROJECT_ID}`
- MODEL_GCP_REGION: A region the AI model is available in. For example `us-east5` works for [Claude](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#regions).
- YOUR_GCP_PROJECT_ID: Get your project id from the [Google cloud dashboard](https://console.cloud.google.com/home/dashboard)
4. Create a new [Chat Model](http://localhost:42110/server/admin/database/chatmodel/add) on your Khoj admin panel.
- **Name**: `claude-3-7-sonnet@20250219`. Any Claude or Gemini model on Vertex's Model Garden should work.
- **Model Type**: `Anthropic` or `Google`
- **Ai Model API**: *the Google Vertex Ai Model API you created in step 3*
- **Max prompt size**: `60000` (replace with the max prompt size of your model)
- **Tokenizer**: *Do not set*
5. Select the chat model on [your settings page](http://localhost:42110/settings) and start a conversation.
## Troubleshooting & gcp AI Tips
- Permission Denied?
Ensure your service account has the `Vertex AI User` role and that the API is enabled in your GCP project.
- Region Errors?
Double-check that the model you're trying to use is supported in your selected region. Some Claude or Gemini models are restricted to specific zones like `us-east5` or `us-central1`.
- Prompt Size Limitations
The "Max prompt size" should align with the limits defined in the model documentation. Exceeding it can silently fail or truncate inputs.
- Testing the API Key
Before adding it to Khoj, you can verify that your key works by making a simple curl request to Vertex AI. This helps debug auth issues early.
- Use Environment Variables
For better security, consider using environment variables to manage sensitive keys and inject them at runtime during base64 encoding.
If you encounter any issues, the [Khoj Discord](https://discord.gg/BDgyabRM6e) is a great place to ask for help!

25
documentation/docs/advanced/litellm.md
View File

@@ -21,17 +21,14 @@ Using LiteLLM with Khoj makes it possible to turn any LLM behind an API into you
export MISTRAL_API_KEY=<MISTRAL_API_KEY>
litellm --model mistral/mistral-tiny --drop_params
```
3. Create a new [OpenAI Processor Conversation Config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) on your Khoj admin panel
- Name: `proxy-name`
- Api Key: `any string`
- Api Base Url: **URL of your Openai Proxy API**
4. Create a new [Chat Model Option](http://localhost:42110/server/admin/database/chatmodeloptions/add) on your Khoj admin panel.
- Name: `llama3.1` (replace with the name of your local model)
- Model Type: `Openai`
- Openai Config: `<the proxy config you created in step 3>`
- Max prompt size: `20000` (replace with the max prompt size of your model)
- Tokenizer: *Do not set for OpenAI, Mistral, Llama3 based models*
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
- Default model: `<name of chat model option you created in step 4>`
- Summarizer model: `<name of chat model option you created in step 4>`
6. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.
3. Create a new [API Model API](http://localhost:42110/server/admin/database/aimodelapi/add) on your Khoj admin panel
- **Name**: `litellm`
- **Api Key**: `any string`
- **Api Base Url**: `<URL of your Openai Proxy API>`
4. Create a new [Chat Model](http://localhost:42110/server/admin/database/chatmodel/add) on your Khoj admin panel.
- **Name**: `llama3.1` (replace with the name of your local model)
- **Model Type**: `Openai`
- **Ai Model Api**: *the litellm Ai Model API you created in step 3*
- **Max prompt size**: `20000` (replace with the max prompt size of your model)
- **Tokenizer**: *Do not set for OpenAI, Mistral, Llama3 based models*
5. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.

29
documentation/docs/advanced/lmstudio.md
View File

@@ -1,4 +1,8 @@
# LM Studio
:::warning[Unsupported]
Khoj does not work with LM Studio anymore. Khoj leverages [json mode](https://platform.openai.com/docs/guides/structured-outputs#json-mode) extensively but LMStudio's API seems to have dropped support for json mode. [1](https://x.com/lmstudio/status/1770135858709975547), [2](https://lmstudio.ai/docs/api/structured-output)
:::
:::info
This is only helpful for self-hosted users. If you're using [Khoj Cloud](https://app.khoj.dev), you're limited to our first-party models.
:::
@@ -14,17 +18,14 @@ LM Studio can expose an [OpenAI API compatible server](https://lmstudio.ai/docs/
## Setup
1. Install [LM Studio](https://lmstudio.ai/) and download your preferred Chat Model
2. Go to the Server Tab on LM Studio, Select your preferred Chat Model and Click the green Start Server button
3. Create a new [OpenAI Processor Conversation Config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) on your Khoj admin panel
- Name: `proxy-name`
- Api Key: `any string`
- Api Base Url: `http://localhost:1234/v1/` (default for LMStudio)
4. Create a new [Chat Model Option](http://localhost:42110/server/admin/database/chatmodeloptions/add) on your Khoj admin panel.
- Name: `llama3.1` (replace with the name of your local model)
- Model Type: `Openai`
- Openai Config: `<the proxy config you created in step 3>`
- Max prompt size: `20000` (replace with the max prompt size of your model)
- Tokenizer: *Do not set for OpenAI, mistral, llama3 based models*
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
- Default model: `<name of chat model option you created in step 4>`
- Summarizer model: `<name of chat model option you created in step 4>`
6. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.
3. Create a new [AI Model API](http://localhost:42110/server/admin/database/aimodelapi/add/) on your Khoj admin panel
- **Name**: `lmstudio`
- **Api Key**: `any string`
- **Api Base Url**: `http://localhost:1234/v1/` (default for LMStudio)
4. Create a new [Chat Model](http://localhost:42110/server/admin/database/chatmodel/add) on your Khoj admin panel.
- **Name**: `llama3.1` (replace with the name of your local model)
- **Model Type**: `Openai`
- **Ai Model Api**: *the lmstudio Ai Model Api you created in step 3*
- **Max prompt size**: `20000` (replace with the max prompt size of your model)
- **Tokenizer**: *Do not set for OpenAI, mistral, llama3 based models*
5. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.

36
documentation/docs/advanced/ollama.md
View File

@@ -1,36 +0,0 @@
# Ollama
:::info
This is only helpful for self-hosted users. If you're using [Khoj Cloud](https://app.khoj.dev), you're limited to our first-party models.
:::
:::info
Khoj natively supports local LLMs [available on HuggingFace in GGUF format](https://huggingface.co/models?library=gguf). Using an OpenAI API proxy with Khoj maybe useful for ease of setup, trying new models or using commercial LLMs via API.
:::
Ollama allows you to run [many popular open-source LLMs](https://ollama.com/library) locally from your terminal.
For folks comfortable with the terminal, Ollama's terminal based flows can ease setup and management of chat models.
Ollama exposes a local [OpenAI API compatible server](https://github.com/ollama/ollama/blob/main/docs/openai.md#models). This makes it possible to use chat models from Ollama to create your personal AI agents with Khoj.
## Setup
1. Setup Ollama: https://ollama.com/
2. Start your preferred model with Ollama. For example,
```bash
ollama run llama3.1
```
3. Create a new [OpenAI Processor Conversation Config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) on your Khoj admin panel
- Name: `ollama`
- Api Key: `any string`
- Api Base Url: `http://localhost:11434/v1/` (default for Ollama)
4. Create a new [Chat Model Option](http://localhost:42110/server/admin/database/chatmodeloptions/add) on your Khoj admin panel.
- Name: `llama3.1` (replace with the name of your local model)
- Model Type: `Openai`
- Openai Config: `<the ollama config you created in step 3>`
- Max prompt size: `20000` (replace with the max prompt size of your model)
5. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
- Default model: `<name of chat model option you created in step 4>`
- Summarizer model: `<name of chat model option you created in step 4>`
6. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.
That's it! You should now be able to chat with your Ollama model from Khoj. If you want to add additional models running on Ollama, repeat step 6 for each model.

78
documentation/docs/advanced/ollama.mdx Normal file
View File

@@ -0,0 +1,78 @@
# Ollama
```mdx-code-block
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
```
:::info
This is only helpful for self-hosted users. If you're using [Khoj Cloud](https://app.khoj.dev), you can use our first-party supported models.
:::
:::info
Khoj can directly run local LLMs [available on HuggingFace in GGUF format](https://huggingface.co/models?library=gguf). The integration with Ollama is useful to run Khoj on Docker and have the chat models use your GPU or to try new models via CLI.
:::
Ollama allows you to run [many popular open-source LLMs](https://ollama.com/library) locally from your terminal.
For folks comfortable with the terminal, Ollama's terminal based flows can ease setup and management of chat models.
Ollama exposes a local [OpenAI API compatible server](https://github.com/ollama/ollama/blob/main/docs/openai.md#models). This makes it possible to use chat models from Ollama with Khoj.
## Setup
:::info
Restart your Khoj server after first run or update to the settings below to ensure all settings are applied correctly.
:::
<Tabs groupId="type" queryString>
<TabItem value="first-run" label="First Run">
<Tabs groupId="server" queryString>
<TabItem value="docker" label="Docker">
1. Setup Ollama: https://ollama.com/
2. Download your preferred chat model with Ollama. For example,
```bash
ollama pull llama3.1
```
3. Uncomment `OPENAI_BASE_URL` environment variable in your downloaded Khoj [docker-compose.yml](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml#:~:text=OPENAI_BASE_URL)
4. Start Khoj docker for the first time to automatically integrate and load models from the Ollama running on your host machine
```bash
# run below command in the directory where you downloaded the Khoj docker-compose.yml
docker-compose up
```
</TabItem>
<TabItem value="pip" label="Pip">
1. Setup Ollama: https://ollama.com/
2. Download your preferred chat model with Ollama. For example,
```bash
ollama pull llama3.1
```
3. Set `OPENAI_BASE_URL` environment variable to `http://localhost:11434/v1/` in your shell before starting Khoj for the first time
```bash
export OPENAI_BASE_URL="http://localhost:11434/v1/"
khoj --anonymous-mode
```
</TabItem>
</Tabs>
</TabItem>
<TabItem value="update" label="Update">
1. Setup Ollama: https://ollama.com/
2. Download your preferred chat model with Ollama. For example,
```bash
ollama pull llama3.1
```
3. Create a new [AI Model API](http://localhost:42110/server/admin/database/aimodelapi/add) on your Khoj admin panel
- **Name**: `ollama`
- **Api Key**: `any string`
- **Api Base Url**: `http://localhost:11434/v1/` (default for Ollama)
4. Create a new [Chat Model](http://localhost:42110/server/admin/database/chatmodel/add) on your Khoj admin panel.
- **Name**: `llama3.1` (replace with the name of your local model)
- **Model Type**: `Openai`
- **AI Model API**: *the ollama AI Model API you created in step 3*
- **Max prompt size**: `20000` (replace with the max prompt size of your model)
5. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.
If you want to add additional models running on Ollama, repeat step 4 for each model.
</TabItem>
</Tabs>
That's it! You should now be able to chat with your Ollama model from Khoj.

39
documentation/docs/advanced/tailscale.md Normal file
View File

@@ -0,0 +1,39 @@
# Tailscale
:::info
This is only helpful for secure cross-device access to **self-hosted** Khoj. You **do not** need this if you're using [Khoj Cloud](https://app.khoj.dev).
:::
[Tailscale](https://tailscale.com) simplifies creating a private VPN using [Wireguard](https://www.wireguard.com/) and OAuth. So you can host and access services on your devices from anywhere.
The instructions below are one way to simply and securely access your self-hosted Khoj from your phone, laptop etc.
### Minimal Setup
1. Setup khoj on your preferred machine following the [standard steps](/get-started/setup)
2. Sign-up to [Tailscale](https://tailscale.com) and install the app on machines you want to access Khoj from. This usually includes your khoj server, your phone, laptop. Note the tailscale i.p of your khoj server.
3. Start khoj on your server by including the flag `--host <your_server_tailscale_ip>`
4. Open `http://<your_server_tailscale_ip>:42110` to access khoj from any device on your tailscale network!
### HTTPS Certificate
:::info
Tailscale uses Wireguard to encrypt and route traffic between your machines. So HTTPS isn't required with Tailscale for secure access. HTTPS with Tailscale is only useful for browsers to not complain about security and block certain features like clipboard access unless HTTPS is enabled.
:::
1. Enable [MagicDNS](https://tailscale.com/kb/1081/magicdns#enabling-magicdns) and [HTTPS](https://tailscale.com/kb/1153/enabling-https) toggle on your tailscale admin console [DNS](https://login.tailscale.com/admin/dns) page. Note your unique tailscale domain name (usually ends with .ts.net)
2. Create an https certificate for your Khoj server by running the following command:
```bash
# Assuming the server is named, `server` and your tailnet is `black-forest.ts.net`
# Note path of the .crt and .key files generated
tailscale cert server.black-forest.ts.net
```
3. Start khoj to be served via https on standard port
```bash
sudo KHOJ_DOMAIN=server.black-forest.ts.net \
khoj \
--sslcert /path/to/your/tailscale.crt \
--sslkey path/to/your/tailscale.key \
--host=server.black-forest.ts.net \
--port 443
```
4. You should now be able to access khoj on `https://server.black-forest.ts.net` from any device on your private tailscale network!

31
documentation/docs/advanced/use-openai-proxy.md
View File

@@ -11,8 +11,8 @@ This is only helpful for self-hosted users. If you're using [Khoj Cloud](https:/
Khoj natively supports local LLMs [available on HuggingFace in GGUF format](https://huggingface.co/models?library=gguf). Using an OpenAI API proxy with Khoj maybe useful for ease of setup, trying new models or using commercial LLMs via API.
:::
Khoj can use any OpenAI API compatible server including [Ollama](/advanced/ollama), [LMStudio](/advanced/lmstudio) and [LiteLLM](/advanced/litellm).
Configuring this allows you to use non-standard, open or commercial, local or hosted LLM models for Khoj
Khoj can use any OpenAI API compatible server including local providers like [Ollama](/advanced/ollama), [LMStudio](/advanced/lmstudio) and [LiteLLM](/advanced/litellm) and commercial providers like [HuggingFace](https://huggingface.co/docs/api-inference/tasks/chat-completion#using-the-api), [OpenRouter](https://openrouter.ai/docs/quick-start) etc.
Configuring this allows you to use non-standard, open or commercial, local or hosted LLM models for Khoj.
Combine them with Khoj can turn your favorite LLM into an AI agent. Allowing you to chat with your docs, find answers from the internet, build custom agents and run automations.
@@ -20,18 +20,15 @@ For specific integrations, see our [Ollama](/advanced/ollama), [LMStudio](/advan
## General Setup
1. Start your preferred OpenAI API compatible app
2. Create a new [OpenAI Processor Conversation Config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) on your Khoj admin panel
- Name: `any name`
- Api Key: `any string`
- Api Base Url: **URL of your Openai Proxy API**
3. Create a new [Chat Model Option](http://localhost:42110/server/admin/database/chatmodeloptions/add) on your Khoj admin panel.
- Name: `llama3` (replace with the name of your local model)
- Model Type: `Openai`
- Openai Config: `<the proxy config you created in step 2>`
- Max prompt size: `2000` (replace with the max prompt size of your model)
- Tokenizer: *Do not set for OpenAI, mistral, llama3 based models*
4. Create a new [Server Chat Setting](http://localhost:42110/server/admin/database/serverchatsettings/add/) on your Khoj admin panel
- Default model: `<name of chat model option you created in step 3>`
- Summarizer model: `<name of chat model option you created in step 3>`
5. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.
1. Start your preferred OpenAI API compatible app locally or get API keys from commercial AI model providers.
3. Create a new [API Model API](http://localhost:42110/server/admin/database/aimodelapi/add) on your Khoj admin panel
- **Name**: `any name`
- **Api Key**: `any string`
- **Api Base Url**: *The URL of your Openai Compatible API*
3. Create a new [Chat Model](http://localhost:42110/server/admin/database/chatmodel/add) on your Khoj admin panel.
- **Name**: `llama3` (replace with the name of your local model)
- **Model Type**: `Openai`
- **Ai Model Api**: *The AI Model API you created in step 2*
- **Max prompt size**: `2000` (replace with the max prompt size of your model)
- **Tokenizer**: *Do not set for OpenAI, mistral, llama3 based models*
4. Go to [your config](http://localhost:42110/settings) and select the model you just created in the chat model dropdown.

38
documentation/docs/clients/desktop.md
View File

@@ -4,22 +4,13 @@ sidebar_position: 1
# Desktop
> Query your Second Brain from your machine
> Upload your knowledge base to Khoj and chat with your whole corpus
Use the Desktop app to chat and search with Khoj.
You can also share your files, folders with Khoj using the app.
## Companion App
Share your files, folders with Khoj using the app.
Khoj will keep these files in sync to provide contextual responses when you search or chat.
## Features
- **Chat**
- **Faster answers**: Find answers quickly, from your private notes or the public internet
- **Assisted creativity**: Smoothly weave across retrieving answers and generating content
- **Iterative discovery**: Iteratively explore and re-discover your notes
- **Quick access**: Use [Khoj Mini](/features/khoj_mini) on the desktop to quickly pull up a mini chat module for quicker answers
- **Search**
- **Natural**: Advanced natural language understanding using Transformer based ML Models
- **Incremental**: Incremental search for a fast, search-as-you-type experience
## Setup
:::info[Self Hosting]
If you are self-hosting the Khoj server, update the *Settings* page on the Khoj Desktop app to:
@@ -34,7 +25,20 @@ If you are self-hosting the Khoj server, update the *Settings* page on the Khoj
4. [Optional] Add any files, folders you'd like Khoj to be aware of on the *Settings* page and Click *Save*.
These files and folders will be automatically kept in sync for you
## Interface
| Chat | Search |
|:----:|:------:|
| ![](/img/khoj_chat_on_desktop.png) | ![](/img/khoj_search_on_desktop.png) |
# Main App
You can also install the Khoj application on your desktop as a progressive web app.
1. Open the [Khoj Web App](https://app.khoj.dev) in Chrome.
2. Click on the install button in the address bar to install the app. You must be logged into your Chrome browser for this to work.
![progressive web app install icon](/img/pwa_install_desktop.png)
Alternatively, you can also install using this route:
1. Open the three-dot menu in the top right corner of the browser.
2. Go to 'Cast, Save, and Share' option.
3. Click on the "Open in Khoj" option.
![progressive web app install route](/img/chrome_pwa_alt.png)

2
documentation/docs/clients/emacs.md
View File

@@ -114,7 +114,7 @@ This feature finds entries similar to the one you are currently on.
2. Hit `C-c s f` (or `M-x khoj RET f`) to find similar entries
### Advanced Usage
- Add [query filters](https://github.com/khoj-ai/khoj/#query-filters) during search to narrow down results further
- Add [query filters](/miscellaneous/query-filters) during search to narrow down results further
e.g. `What is the meaning of life? -"god" +"none" dt>"last week"`
- Use `C-c C-o 2` to open the current result at cursor in its source org file

2
documentation/docs/clients/obsidian.md
View File

@@ -48,7 +48,7 @@ To see other notes similar to the current one, run *Khoj: Find Similar Notes* fr
### Search
Run *Khoj: Search* from the [Command Palette](https://help.obsidian.md/Plugins/Command+palette)
See [Khoj Search](/features/search) for more details. Use [query filters](/miscellaneous/advanced#query-filters) to limit entries to search
See [Khoj Search](/features/search) for more details. Use [query filters](/miscellaneous/query-filters) to limit entries to search
[search_demo](https://user-images.githubusercontent.com/6413477/218801155-cd67e8b4-a770-404a-8179-d6b61caa0f93.mp4 ':include :type=mp4')

6
documentation/docs/clients/whatsapp.md
View File

@@ -6,13 +6,13 @@ sidebar_position: 5
> Query your Second Brain from WhatsApp
Text [+1 (848) 800 4242](https://wa.me/18488004242) or scan the QQ code below on your phone to chat with Khoj on WhatsApp.
Text [+1 (848) 800 4242](https://wa.me/18488004242) or scan the QR code below on your phone to chat with Khoj on WhatsApp.
Without any desktop clients, you can start chatting with Khoj on WhatsApp. Bear in mind you do need one of the desktop clients in order to share and sync your data with Khoj. The WhatsApp AI bot will work right away for answering generic queries and using Khoj in default mode.
In order to use Khoj on WhatsApp with your own data, you need to setup a Khoj Cloud account and connect your WhatsApp account to it. This is a one time setup and you can do it from the [Khoj Cloud config page](https://app.khoj.dev/settings).
If you hit usage limits for the WhatsApp bot, upgrade to [a paid plan](https://khoj.dev/pricing) on Khoj Cloud.
If you hit usage limits for the WhatsApp bot, upgrade to [a paid plan](https://khoj.dev/#pricing) on Khoj Cloud.
<img src="https://khoj-web-bucket.s3.amazonaws.com/khojwhatsapp.png" alt="WhatsApp QR Code" width="300" height="300" />
@@ -27,4 +27,4 @@ We have more commands under development, including `/share` to uploading documen
## Source Code
You can find all of the code for the WhatsApp bot in the the [flint repository](https://github.com/khoj-ai/flint). As all of our code, it is open source and you can contribute to it.
You can find all of the code for the WhatsApp bot in the [flint repository](https://github.com/khoj-ai/flint). As all of our code, it is open source and you can contribute to it.

31
documentation/docs/contributing/development.mdx
View File

@@ -4,7 +4,7 @@ sidebar_position: 0
# Development
Welcome to the development docs of Khoj! Thanks for you interesting in being a contributor ❤️. Open source contributors are a corner-store of the Khoj community. We welcome all contributions, big or small.
Welcome to the development docs of Khoj! Thanks for your interest in being a contributor ❤️. Open source contributors are a corner-store of the Khoj community. We welcome all contributions, big or small.
To get started with contributing, check out the official GitHub docs on [contributing to an open-source project](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project).
@@ -30,7 +30,7 @@ git clone https://github.com/khoj-ai/khoj && cd khoj
python3 -m venv .venv && source .venv/bin/activate
# For MacOS or zsh users run this
pip install -e '.[dev]'
uv sync --all-extras
```
</TabItem>
<TabItem value="windows" label="Windows">
@@ -42,7 +42,7 @@ git clone https://github.com/khoj-ai/khoj && cd khoj
python3 -m venv .venv && .venv\Scripts\activate
# Install Khoj for Development
pip install -e '.[dev]'
uv sync --all-extras
```
</TabItem>
<TabItem value="linux" label="Linux">
@@ -54,7 +54,7 @@ git clone https://github.com/khoj-ai/khoj && cd khoj
python3 -m venv .venv && source .venv/bin/activate
# Install Khoj for Development
pip install -e '.[dev]'
uv sync --all-extras
```
</TabItem>
</Tabs>
@@ -102,7 +102,19 @@ sudo -u postgres createdb khoj --password
</TabItem>
</Tabs>
#### 3. Run
#### 3. Build the front-end assets
```shell
cd src/interface/web/
bun install
bun export
```
You can optionally use `bun dev` to start a development server for the front-end which will be available at http://localhost:3000. This is especially useful if you're making changes to the front-end code, but not necessary for running Khoj. Note that streaming does not work on the dev server due to how it is handled with SSR in Next.js.
Always run `bun export` to test your front-end changes on http://localhost:42110 before creating a PR.
#### 4. Run
1. Start Khoj
```bash
khoj -vv
@@ -117,7 +129,7 @@ sudo -u postgres createdb khoj --password
- Try reactivating the virtual environment and rerunning the `khoj` command.
- If it still doesn't work repeat the installation process.
2. Python Package Missing
- Use `pip install xxx` and try running the `khoj` command.
- Use `uv add xxx` and try running the `khoj` command.
3. Command `createdb` Not Recognized
- make sure path to postgres binaries is included in environment variables. It usually looks something like
```
@@ -179,18 +191,19 @@ In whichever clients you're using for testing, you'll need to update the server
## Validate
### Before Making Changes
1. Install Git Hooks for Validation
1. Install Khoj. Setup Git Hooks for Validation
```shell
./scripts/dev_setup.sh
```
- This ensures standard code formatting fixes and other checks run automatically on every commit and push
- Installs the khoj server and web app by default. Use the `--full` flag to install the Khoj Desktop and Obsidian client apps as well.
- Git hooks format code, check types automatically on every commit and push respectively.
- Note 1: If [pre-commit](https://pre-commit.com/#intro) didn't already get installed, [install it](https://pre-commit.com/#install) via `pip install pre-commit`
- Note 2: To run the pre-commit changes manually, use `pre-commit run --hook-stage manual --all` before creating PR
### Before Creating PR
:::tip[Note]
You should be in an active virtual environment for Khoj in order to run the unit tests and linter.
You should be in an active virtual environment for Khoj in order to run the unit tests and linter. The `dev_setup.sh` script wil automatically create and activate it for you.
:::
1. Ensure that you have a [Github Issue](https://github.com/khoj-ai/khoj/issues) that can be linked to the PR. If not, create one. Make sure you've tagged one of the maintainers to the issue. This will ensure that the maintainers are notified of the PR and can review it. It's best discuss the code design on an existing issue or Discord thread before creating a PR. This helps get your PR merged faster.

8
documentation/docs/data-sources/github_integration.md
View File

@@ -1,6 +1,10 @@
# Github integration
The Github integration allows you to index as many repositories as you want. It's currently default configured to index Issues, Commits, and all Markdown/Org files in each repository. For large repositories, this takes a fairly long time, but it works well for smaller projects.
:::warning[Unmaintained]
The Github integration is not maintained. We are considering deprecating it. It doesn't seem used by many folks and its cumbersome for us to maintain.
:::
The Github integration allows you to index as many repositories as you want. It's currently default configured to index all Markdown/Org/Text files in each repository. For large repositories, this takes a fairly long time, but it works well for smaller projects.
# Configure your settings
@@ -9,6 +13,6 @@ The Github integration allows you to index as many repositories as you want. It'
## Use the Github plugin
1. Generate a [classic PAT (personal access token)](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) from [Github](https://github.com/settings/tokens) with `repo` and `admin:org` scopes at least.
2. Navigate to [https://app.khoj.dev/settings#github](https://app.khoj.dev/settings#github) to configure your Github settings. Enter in your PAT, along with details for each repository you want to index.
2. Navigate to [https://app.khoj.dev/settings#github](https://app.khoj.dev/settings/content/github) to configure your Github settings. Enter in your PAT, along with details for each repository you want to index.
3. Click `Save`. Go back to the settings page and click `Configure`.
4. Go to [https://app.khoj.dev/](https://app.khoj.dev/) and start searching!

2
documentation/docs/data-sources/notion_integration.md
View File

@@ -16,4 +16,4 @@ Go to https://app.khoj.dev/settings to connect your Notion workspace(s) to Khoj.
4. In the first step, you generated an API key. Use the newly generated API Key in your Khoj settings, by default at [http://localhost:42110/settings#notion](http://localhost:42110/settings#notion). Click `Save`.
5. Click `Configure` in http://localhost:42110/settings to index your Notion workspace(s).
That's it! You should be ready to start searching and chatting. Make sure you've configured your [chat settings](/get-started/setup#2-configure).
That's it! You should be ready to start searching and chatting. Make sure you've configured your [chat settings](/get-started/setup#use-khoj).

1
documentation/docs/data-sources/share_your_data.md
View File

@@ -9,6 +9,7 @@ keywords: ["upload data", "upload files", "share data", "share files", "pdf ai",
There are several ways you can get started with sharing your data with the Khoj AI.
- Drag and drop your documents via [the web UI](/clients/web/#upload-documents). This is best if you have a one-off document you need to interact with.
- You can use the [search page](https://app.khoj.dev/search) to upload documents that become indexed & searchable by your second brain. Click on "Add Documents" to get started.
- Use the desktop app to [upload and sync your documents](/clients/desktop). This is best if you have a lot of documents on your computer or you need the docs to stay in sync.
- Setup the sync options for either [Obsidian](/clients/obsidian) or [Emacs](/clients/emacs) to automatically sync your documents with Khoj. This is best if you are already using these tools and want to leverage Khoj's AI capabilities.
- Configure your [Notion](/data-sources/notion_integration) or [Github](/data-sources/github_integration) to sync with Khoj. By providing your credentials, you can keep the data synced in the background.

4
documentation/docs/features/all-features.md
View File

@@ -17,7 +17,7 @@ Khoj supports a variety of features, including search and chat with a wide range
- **Works online or offline**: Chat using online or offline AI chat models
#### General
- **Cloud or Self-Host**: Use [cloud](https://app.khoj.dev/login) to use Khoj anytime from anywhere or [self-host](/get-started/setup) for privacy
- **Cloud or Self-Host**: Use [cloud](https://app.khoj.dev) to use Khoj anytime from anywhere or [self-host](/get-started/setup) for privacy
- **Natural**: Advanced natural language understanding using Transformer based ML Models
- **Pluggable**: Modular architecture makes it easy to plug in new data sources, frontends and ML models
- **Multiple Sources**: Index your Org-mode, Markdown, PDF, plaintext files, Github repos and Notion pages
@@ -29,7 +29,7 @@ Khoj is available as a [Desktop app](/clients/desktop), [Emacs package](/clients
![](/img/khoj_clients.svg ':size=400px')
### Supported Data Sources
Khoj can understand your word, PDF, org-mode, markdown, plaintext files, [Github projects](/data-sources/github_integration) and [Notion pages](/data-sources/notion_integration).
Khoj can understand your word, PDF, org-mode, markdown, plaintext files, and [Notion pages](/data-sources/notion_integration).
![](/img/khoj_datasources.svg ':size=200px')

2
documentation/docs/features/automations.md
View File

@@ -9,6 +9,6 @@ Khoj uses your local time zone for scheduling. You can configure, share and dele
Self hosting users need to setup Resend and [Authentication](/advanced/authentication) setup to have Khoj send automation emails.
:::
| [Create Automation](https://app.khoj.dev/automations?subject=Weekly%20Newsletter&query=Compile%20a%20message%20including:%201.%20A%20recap%20of%20news%20from%20last%20week%202.%20An%20at-home%20workout%20I%20can%20do%20before%20work%203.%20A%20quote%20to%20inspire%20me%20for%20the%20week%20ahead&crontime=00%209%20*%20*%201) | Recieve Automation Email |
| [Create Automation](https://app.khoj.dev/automations?subject=Weekly%20Newsletter&query=Compile%20a%20message%20including:%201.%20A%20recap%20of%20news%20from%20last%20week%202.%20An%20at-home%20workout%20I%20can%20do%20before%20work%203.%20A%20quote%20to%20inspire%20me%20for%20the%20week%20ahead&crontime=00%209%20*%20*%201) | Receive Automation Email |
|--------|-------|
| <img src="/img/khoj_create_automation.png" alt="create automation" width="330" height="500" /> | <img src="/img/khoj_automation_email.png" alt="create automation" width="330" height="660" /> |

8
documentation/docs/features/chat.md
View File

@@ -22,7 +22,7 @@ See [the setup guide](/get-started/setup.mdx) to configure your chat models.
- **On Web**: Open [/chat](https://app.khoj.dev/chat) in your web browser
- **On Obsidian**: Search for *Khoj: Chat* in the [Command Palette](https://help.obsidian.md/Plugins/Command+palette)
- **On Emacs**: Run `M-x khoj <user-query>`
2. Enter your queries to chat with Khoj. Use [slash commands](#commands) and [query filters](/miscellaneous/advanced#query-filters) to change what Khoj uses to respond
2. Enter your queries to chat with Khoj. Use [slash commands](#commands) and [query filters](/miscellaneous/query-filters) to change what Khoj uses to respond
#### Details
@@ -30,7 +30,7 @@ See [the setup guide](/get-started/setup.mdx) to configure your chat models.
2. These notes, the last few messages and associated metadata is passed to the enabled chat model along with your query to generate a response
#### Conversation File Filters
You can use conversation file filters to limit the notes used in the chat response. To do so, use the left panel in the web UI. Alternatively, you can also use [query filters](/miscellaneous/advanced#query-filters) to limit the notes used in the chat response.
You can use conversation file filters to limit the notes used in the chat response. To do so, use the left panel in the web UI. Alternatively, you can also use [query filters](/miscellaneous/query-filters) to limit the notes used in the chat response.
<img src="/img/file_filters_conversation.png" alt="Conversation File Filter" style={{width: '400px'}}/>
@@ -42,4 +42,6 @@ Slash commands allows you to change what Khoj uses to respond to your query
- **/online**: Use online information and incorporate it in the prompt to the LLM to send you a response.
- **/image**: Generate an image in response to your query.
- **/help**: Use /help to get all available commands and general information about Khoj
- **/summarize**: Can be used to summarize 1 selected file filter for that conversation. Refer to [File Summarization](summarization) for details.
- **/diagram**: Generate a diagram in response to your query. This is built on [Excalidraw](https://excalidraw.com/).
- **/code**: Generate and run very simple Python code snippets. Refer to [Code Execution](code_execution) for details.
- **/research**: Go deeper in a topic for more accurate, in-depth responses.

40
documentation/docs/features/code_execution.md Normal file
View File

@@ -0,0 +1,40 @@
---
---
# Code Execution
Khoj can generate and run simple Python code as well. This is useful if you want to have Khoj do some data analysis, generate plots and reports. LLMs by default aren't skilled at complex quantitative tasks. Code generation & execution can come in handy for such tasks.
Khoj automatically infers when to use the code tool. You can also tell it explicitly to use the code tool or use the `/code` [slash command](https://docs.khoj.dev/features/chat/#commands) in your chat.
## Setup (Self-Hosting)
### Terrarium Sandbox
Use [Cohere's Terrarium](https://github.com/cohere-ai/cohere-terrarium) to host the code sandbox locally on your machine for free.
To run with Docker, use our [docker-compose.yml](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml) to automatically setup the Terrarium code sandbox, or start it manually like this:
```bash
docker pull ghcr.io/khoj-ai/terrarium:latest
docker run -d -p 8080:8080 ghcr.io/khoj-ai/terrarium:latest
```
To run from source, check [these instructions](https://github.com/khoj-ai/cohere-terrarium?tab=readme-ov-file#development).
#### Verify
Verify that it's running, by evaluating a simple Python expression:
```bash
curl -X POST -H "Content-Type: application/json" \
--url http://localhost:8080 \
--data-raw '{"code": "1 + 1"}' \
--no-buffer
```
### E2B Sandbox
[E2B](https://e2b.dev/) allows Khoj to run code on a remote but versatile sandbox with support for more python libraries. This is [not free](https://e2b.dev/pricing).
To have Khoj use E2B as the code sandbox:
1. Generate an API key on [their dashboard](https://e2b.dev/dashboard).
2. Set the `E2B_API_KEY` environment variable to it on the machine running your Khoj server.
- When using our [docker-compose.yml](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml), uncomment and set the `E2B_API_KEY` env var in the `docker-compose.yml` file.
3. Now restart your Khoj server to switch to using the E2B code sandbox.

17
documentation/docs/features/image_generation.md
View File

@@ -10,6 +10,17 @@ To generate images, you just need to provide a prompt to Khoj in which the image
## Setup (Self-Hosting)
Right now, we only support integration with OpenAI's DALL-E. You need to have an OpenAI API key to use this feature. Here's how you can set it up:
1. Setup your OpenAI API key. See instructions [here](/get-started/setup#2-configure)
2. Create a text to image config at http://localhost:42110/server/admin/database/texttoimagemodelconfig/. We recommend the value `dall-e-3`.
You have a couple of image generation options.
### Image Generation Models
We support most state of the art image generation models, including Ideogram, Flux, and Stable Diffusion. These will run using [Replicate](https://replicate.com). Here's how to set them up:
1. Get a Replicate API key [here](https://replicate.com/account/api-tokens).
2. Create a new [Text to Image Model](http://localhost:42110/server/admin/database/texttoimagemodelconfig/). Set the `type` to `Replicate`. Use any of the model names you see [on this list](https://replicate.com/pricing#image-models). We recommend the `model name` `black-forest-labs/flux-1.1-pro` from [Replicate](https://replicate.com/black-forest-labs/flux-1.1-pro).
### OpenAI
1. Get [an OpenAI API key](https://platform.openai.com/settings/organization/api-keys).
2. Setup your OpenAI API key, if you haven't already. See instructions [here](/get-started/setup#add-chat-models)
3. Create a text to image config at http://localhost:42110/server/admin/database/texttoimagemodelconfig/. Use `model name` `dall-e-3` to use openai for image generation. Make sure to set the `Ai model api` field to the OpenAI AI model api you setup in step 2.

17
documentation/docs/features/online_search.md
View File

@@ -14,8 +14,19 @@ Try it out yourself! https://app.khoj.dev
## Self-Hosting
Online search works out of the box even when self-hosting. Khoj uses [JinaAI's reader API](https://jina.ai/reader/) to search online and read webpages by default. No API key setup is necessary.
### Search
To improve online search, set the `SERPER_DEV_API_KEY` environment variable to your [Serper.dev](https://serper.dev/) API key. These search results include additional context like answer box, knowledge graph etc.
Online search can work even with self-hosting! You have a few options:
For advanced webpage reading, set the `OLOSTEP_API_KEY` environment variable to your [Olostep](https://www.olostep.com/) API key. This has a higher success rate at reading webpages than the default webpage reader.
- If you're using Docker, online search should work out of the box with [searxng](https://github.com/searxng/searxng) using our standard `docker-compose.yml`.
- To get production-grade, fast online search, set the `SERPER_DEV_API_KEY` environment variable to your [Serper.dev](https://serper.dev/) API key. These search results include additional context like answer box, knowledge graph etc.
- To use open, self-hostable search provider, set the `FIRECRAWL_API_KEY` environment variable to your [Firecrawl](https://firecrawl.dev) API key. These search results do not scrape social media results.
- To use Exa search provider, set the `EXA_API_KEY` environment variable to your [Exa](https://exa.ai) API key.
### Webpage Reading
Out of the box, you **don't have to do anything to enable webpage reading**. Khoj will automatically read webpages by using the `requests` library. To get faster, more readable webpages for Khoj, you can use the following options:
- For open, self-hostable webpage reader, you can use [Firecrawl](https://www.firecrawl.dev/). Create a new [Webscraper](http://localhost:42110/server/admin/database/webscraper/add/). Set your Firecrawl API key to the Api Key field, and set the type to Firecrawl.
- For advanced webpage reading, you can use [Olostep](https://www.olostep.com/). This can read a wider variety of webpages. Create a new [Webscraper](http://localhost:42110/server/admin/database/webscraper/add/). Set your Olostep API key to the Api Key field, and set the type to Olostep.
- For fast webpage reading, you can use [Exa](https://exa.ai). Create a new [Webscraper](http://localhost:42110/server/admin/database/webscraper/add/). Set your Exa API key to the Api Key field, and set the type to Exa.

36
documentation/docs/features/search.md
View File

@@ -11,7 +11,41 @@ Take advantage of super fast search to find relevant notes and documents from yo
- **On Web**: Open https://app.khoj.dev/ in your web browser
- **On Obsidian**: Click the *Khoj search* icon 🔎 on the [Ribbon](https://help.obsidian.md/User+interface/Workspace/Ribbon) or Search for *Khoj: Search* in the [Command Palette](https://help.obsidian.md/Plugins/Command+palette)
- **On Emacs**: Run `M-x khoj <user-query>`
2. Query using natural language to find relevant entries from your knowledge base. Use [query filters](/miscellaneous/advanced#query-filters) to limit entries to search
2. Query using natural language to find relevant entries from your knowledge base. Use [query filters](/miscellaneous/query-filters) to limit entries to search
### Demo
![](/img/search_agents_markdown.png ':size=400px')
### Implementation Overview
A bi-encoder models is used to create meaning vectors (aka vector embeddings) of your documents and search queries.
1. When you sync you documents with Khoj, it uses the bi-encoder model to create and store meaning vectors of (chunks of) your documents
2. When you initiate a natural language search the bi-encoder model converts your query into a meaning vector and finds the most relevant document chunks for that query by comparing their meaning vectors.
3. The slower but higher-quality cross-encoder model is than used to re-rank these documents for your given query.
### Setup (Self-Hosting)
You are **not required** to configure the search model config when self-hosting. Khoj sets up decent default local search model config for general use.
You may want to configure this if you need better multi-lingual search, want to experiment with different, newer models or the default models do not work for your use-case.
You can use bi-encoder models downloaded locally [from Huggingface](https://huggingface.co/models?library=sentence-transformers), served via the [HuggingFace Inference API](https://endpoints.huggingface.co/), OpenAI API, Azure OpenAI API or any OpenAI Compatible API like Ollama, LiteLLM etc. Follow the steps below to configure your search model:
1. Open the [SearchModelConfig](http://localhost:42110/server/admin/database/searchmodelconfig/) page on your Khoj admin panel.
2. Hit the Plus button to add a new model config or click the id of an existing model config to edit it.
3. Set the `biencoder` field to the name of the bi-encoder model supported [locally](https://huggingface.co/models?library=sentence-transformers) or via the API you configure.
4. Set the `Embeddings inference endpoint api key` to your OpenAI API key and `Embeddings inference endpoint type` to `OpenAI` to use an OpenAI embedding model.
5. Also set the `Embeddings inference endpoint` to your Azure OpenAI or OpenAI compatible API URL to use the model via those APIs.
6. Ensure the search model config you want to use is the **only one** that has `name` field set to `default`[^1].
7. Save the search model configs and restart your Khoj server to start using your new, updated search config.
:::info
You will need to re-index all your documents if you want to use a different bi-encoder model.
:::
:::info
You may need to tune the `Bi encoder confidence threshold` field for each bi-encoder to get appropriate number of documents for chat with your Knowledge base.
Confidence here is a normalized measure of semantic distance between your query and documents. The confidence threshold limits the documents returned to chat that fall within the distance specified in this field. It can take values between 0.0 (exact overlap) and 1.0 (no meaning overlap).
:::
[^1]: Khoj uses the first search model config named `default` it finds on startup as the search model config for that session

26
documentation/docs/features/summarization.md
View File

@@ -1,26 +0,0 @@
---
sidebar_position: 5
---
# File Summarization
You can use the `/summarize` command to get Khoj to generate context driven summaries of your documents.
Simply select a single file filter on the left panel menu and then use `/summarize [any context]` and Khoj
will produce a tailored summary of the text.
You can also try a natural language query which include the intent for summary without explicitly using the `/summarize` command.
## Design Diagram
<img src="/img/summarize.jpg" alt="Chat on Web" style={{width: '800px'}}/>
## Example Usage
* `/summarize in a way that can be used as practice questions for a test`
* `/summarize in a way a toddler can understand`
* `/summarize in one paragraph`
Without using the `/summarize` command:
* `create a summary of the document in a way that can be used as practice questions for a test`
* `summarize the document in a way a toddler can understand`
* `generate a one paragraph summary of the document`

4
documentation/docs/features/voice-chat.md
View File

@@ -17,11 +17,11 @@ You can also click on the speaker icon next to any message to hear it out loud.
Voice chat will automatically be configured when you initialize the application. The default configuration will run locally. If you want to use the OpenAI whisper API for voice chat, you can set it up by following these steps:
1. Setup your OpenAI API key. See instructions [here](/get-started/setup#2-configure).
1. Setup your OpenAI API key. See instructions [here](/get-started/setup#add-chat-models).
2. Create a new configuration at http://localhost:42110/server/admin/database/speechtotextmodeloptions/. We recommend the value `whisper-1` and model type `Openai`.
If you want to use the Text to Speech feature, you can set it up by following these steps:
1. Setup your account on [ElevenLabs.io](https://elevenlabs.io/).
2. Configure your API key in your environment variables with the key `ELEVEN_LABS_API_KEY`.
2. (Optional) Create a new [Voice model option](http://localhost:42110/server/admin/database/voicemodeloption/) with a specific voice ID from whichever voice you want to use. You can explore the options [here](https://elevenlabs.io/app/voice-library).
3. (Optional) Create a new [Voice model option](http://localhost:42110/server/admin/database/voicemodeloption/) with a specific voice ID from whichever voice you want to use. You can explore the options [here](https://elevenlabs.io/app/voice-library).

9
documentation/docs/get-started/overview.md
View File

@@ -1,13 +1,10 @@
---
sidebar_position: 0
slug: /
keywords: ["khoj", "khoj ai", "khoj docs", "khoj documentation", "khoj features", "khoj overview", "khoj quickstart", "khoj chat", "khoj search", "khoj cloud", "khoj self-host", "khoj setup", "open source ai", "local llm", "ai copilot", "second brain", "personal ai", "ai search engine"]
keywords: ["khoj", "khoj ai", "khoj docs", "khoj documentation", "khoj features", "khoj overview", "khoj quickstart", "khoj chat", "khoj search", "khoj cloud", "khoj self-host", "khoj setup", "open source ai", "local llm", "ai copilot", "second brain", "personal ai", "ai search engine", "research assistant", "open source llm", "self-host llm", "self-host chatgpt", "ai chatbot", "ai assistant", "ai research assistant", "ai search engine", "ai knowledge base", "ai knowledge graph", "ai personal assistant", "ai second brain", "open source ai assistant"]
---
# Overview
<p align="center"><img src="/img/khoj-logo-sideways-500.png" width="200" alt="Khoj Logo"></img></p>
<div align="center">
<b>Your Second Brain</b>
</div>
@@ -29,9 +26,9 @@ Welcome to the Khoj Docs! This is the best place to get setup and explore Khoj's
- Khoj is an open source, personal AI
- You can [chat](/features/chat) with it about anything. It'll use files you shared with it to respond, when relevant. It can also access information from the public internet.
- Quickly [find](/features/search) relevant notes and documents using natural language
- It understands pdf, plaintext, markdown, org-mode files, [notion pages](/data-sources/notion_integration) and [github repositories](/data-sources/github_integration)
- It understands pdf, plaintext, markdown, org-mode files, and [notion pages](/data-sources/notion_integration).
- Access it from your [Emacs](/clients/emacs), [Obsidian](/clients/obsidian), the [Khoj desktop app](/clients/desktop), or [any web browser](/clients/web)
- Use our [cloud](https://app.khoj.dev/login) instance to access your Khoj anytime from anywhere, [self-host](/get-started/setup) on consumer hardware for privacy
- Use our [cloud](https://app.khoj.dev) instance to access your Khoj anytime from anywhere, [self-host](/get-started/setup) on consumer hardware for privacy
![demo_chat](https://assets.khoj.dev/quadratic_equation_khoj_web.gif)

2
documentation/docs/get-started/privacy-security.md
View File

@@ -17,7 +17,7 @@ Here's what to consider if you're using Khoj, whether self-hosted or on our clou
- If you're self-hosting, you can opt out of telemetry by following [these instructions](/miscellaneous/telemetry).
Self-hosting isn't for everyone, so we've still taken steps to make Khoj privacy-friendly, even if you choose to use our [cloud offering](https://app.khoj.dev/login). Here's what to consider when using Khoj Cloud:
Self-hosting isn't for everyone, so we've still taken steps to make Khoj privacy-friendly, even if you choose to use our [cloud offering](https://app.khoj.dev). Here's what to consider when using Khoj Cloud:
1. Your embeddings are generated by an open source model within our own dedicated endpoint [hosted on AWS with Huggingface](https://huggingface.co/inference-endpoints/dedicated). There's zero persistent memory to the Huggingface Inference endpoints (it's stateless).
1. Your embeddings and the associated raw text are stored in a secure Postgres DB in our private AWS cloud. Your data is sharded on a unique user ID. We store the raw text in your files to improve file syncing and provide context when you chat with Khoj.
1. When you use the single-sign-on option with Google, we only receive your name, a link to your profile photo, and your email address.

207
documentation/docs/get-started/setup.mdx
View File

@@ -18,8 +18,8 @@ import TabItem from '@theme/TabItem';
These are the general setup instructions for self-hosted Khoj.
You can install the Khoj server using either [Docker](?server=docker) or [Pip](?server=pip).
:::info[Offline Model + GPU]
If you want to use the offline chat model and you have a GPU, you should use Installation Option 2 - local setup via the Python package directly. Our Docker image doesn't currently support running the offline chat model on GPU, making inference times really slow.
:::info[First Run]
Restart your Khoj server after the first run to ensure all settings are applied correctly.
:::
<Tabs groupId="server" queryString>
@@ -27,7 +27,29 @@ If you want to use the offline chat model and you have a GPU, you should use Ins
<Tabs groupId="operating-systems" queryString="os">
<TabItem value="macos" label="MacOS">
<h3>Prerequisites</h3>
Install [Docker Desktop](https://docs.docker.com/desktop/install/mac-install/)
<h4>Docker</h4>
- *Option 1*: Click here to install [Docker Desktop](https://docs.docker.com/desktop/install/mac-install/). Make sure you also install the [Docker Compose](https://docs.docker.com/desktop/install/mac-install/) tool.
- *Option 2*: Use [Homebrew](https://brew.sh/) to install Docker and Docker Compose.
```shell
brew install --cask docker
brew install docker-compose
```
<h3>Setup</h3>
1. Download the Khoj docker-compose.yml file [from Github](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml)
```shell
mkdir ~/.khoj && cd ~/.khoj
wget https://raw.githubusercontent.com/khoj-ai/khoj/master/docker-compose.yml
```
2. Configure the environment variables in the `docker-compose.yml`
- Set `KHOJ_ADMIN_PASSWORD`, `KHOJ_DJANGO_SECRET_KEY` (and optionally the `KHOJ_ADMIN_EMAIL`) to something secure. This allows you to customize Khoj later via the admin panel.
- Set `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY` to your API key if you want to use OpenAI, Anthropic or Gemini commercial chat models respectively.
- Uncomment `OPENAI_BASE_URL` to use [Ollama](/advanced/ollama?type=first-run&server=docker#setup) running on your host machine. Or set it to the URL of your OpenAI compatible API like vLLM or [LMStudio](/advanced/lmstudio).
3. Start Khoj by running the following command in the same directory as your docker-compose.yml file.
```shell
cd ~/.khoj
docker-compose up
```
</TabItem>
<TabItem value="windows" label="Windows">
<h3>Prerequisites</h3>
@@ -37,86 +59,59 @@ If you want to use the offline chat model and you have a GPU, you should use Ins
wsl --install
```
2. Install [Docker Desktop](https://docs.docker.com/desktop/install/windows-install/) with **[WSL2 backend](https://docs.docker.com/desktop/wsl/#turn-on-docker-desktop-wsl-2)** (default)
<h3>Setup</h3>
1. Download the Khoj docker-compose.yml file [from Github](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml)
```shell
# Windows users should use their WSL2 terminal to run these commands
mkdir ~/.khoj && cd ~/.khoj
wget https://raw.githubusercontent.com/khoj-ai/khoj/master/docker-compose.yml
```
2. Configure the environment variables in the `docker-compose.yml`
- Set `KHOJ_ADMIN_PASSWORD`, `KHOJ_DJANGO_SECRET_KEY` (and optionally the `KHOJ_ADMIN_EMAIL`) to something secure. This allows you to customize Khoj later via the admin panel.
- Set `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY` to your API key if you want to use OpenAI, Anthropic or Gemini commercial chat models respectively.
- Uncomment `OPENAI_BASE_URL` to use [Ollama](/advanced/ollama) running on your host machine. Or set it to the URL of your OpenAI compatible API like vLLM or [LMStudio](/advanced/lmstudio).
3. Start Khoj by running the following command in the same directory as your docker-compose.yml file.
```shell
# Windows users should use their WSL2 terminal to run these commands
cd ~/.khoj
docker-compose up
```
</TabItem>
<TabItem value="linux" label="Linux">
<h3>Prerequisites</h3>
Install [Docker Desktop](https://docs.docker.com/desktop/install/windows-install/).
Install [Docker Desktop](https://docs.docker.com/desktop/install/linux/).
You can also use your package manager to install Docker Engine & Docker Compose.
<h3>Setup</h3>
1. Download the Khoj docker-compose.yml file [from Github](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml)
```shell
mkdir ~/.khoj && cd ~/.khoj
wget https://raw.githubusercontent.com/khoj-ai/khoj/master/docker-compose.yml
```
2. Configure the environment variables in the `docker-compose.yml`
- Set `KHOJ_ADMIN_PASSWORD`, `KHOJ_DJANGO_SECRET_KEY` (and optionally the `KHOJ_ADMIN_EMAIL`) to something secure. This allows you to customize Khoj later via the admin panel.
- Set `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY` to your API key if you want to use OpenAI, Anthropic or Gemini commercial chat models respectively.
- Uncomment `OPENAI_BASE_URL` to use [Ollama](/advanced/ollama) running on your host machine. Or set it to the URL of your OpenAI compatible API like vLLM or [LMStudio](/advanced/lmstudio).
3. Start Khoj by running the following command in the same directory as your docker-compose.yml file.
```shell
cd ~/.khoj
docker-compose up
```
</TabItem>
</Tabs>
<h3>Setup</h3>
1. Download the Khoj docker-compose.yml file [from Github](https://github.com/khoj-ai/khoj/blob/master/docker-compose.yml)
```shell
# Windows users should use their WSL2 terminal to run these commands
mkdir ~/.khoj && cd ~/.khoj
wget https://raw.githubusercontent.com/khoj-ai/khoj/master/docker-compose.yml
```
2. Configure the environment variables in the docker-compose.yml
- Set `KHOJ_ADMIN_PASSWORD`, `KHOJ_DJANGO_SECRET_KEY` (and optionally the `KHOJ_ADMIN_EMAIL`) to something secure. This allows you to customize Khoj later via the admin panel.
- Set `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY` to your API key if you want to use OpenAI, Anthropic or Gemini chat models respectively.
3. Start Khoj by running the following command in the same directory as your docker-compose.yml file.
```shell
# Windows users should use their WSL2 terminal to run these commands
cd ~/.khoj
docker-compose up
```
:::info[Remote Access]
By default Khoj is only accessible on the machine it is running. To access Khoj from a remote machine see [Remote Access Docs](/advanced/remote).
:::
Your setup is complete once you see `🌖 Khoj is ready to use` in the server logs on your terminal.
Your setup is complete once you see `🌖 Khoj is ready to engage` in the server logs on your terminal.
</TabItem>
<TabItem value="pip" label="Pip">
<h3>1. Install Postgres (with PgVector)</h3>
Khoj uses Postgres DB for all server configuration and to scale to multi-user setups. It uses the pgvector package in Postgres to manage your document embeddings. Both Postgres and pgvector need to be installed for Khoj to work.
<Tabs groupId="operating-systems" queryString="os">
<TabItem value="macos" label="MacOS">
Install [Postgres.app](https://postgresapp.com/). This comes pre-installed with `pgvector` and relevant dependencies.
</TabItem>
<TabItem value="windows" label="Windows">
For detailed instructions and troubleshooting, see [this section](/contributing/development#2-postgres-installation--setup).
1. Use the [recommended installer](https://www.postgresql.org/download/windows/).
2. Follow instructions to [Install PgVector](https://github.com/pgvector/pgvector#windows) in case you need to manually install it. Windows support is experimental for pgvector currently, so we recommend using Docker.
</TabItem>
<TabItem value="linux" label="Linux">
From [official instructions](https://wiki.postgresql.org/wiki/Apt)
</TabItem>
<TabItem value="source" label="From Source">
1. Follow instructions to [Install Postgres](https://www.postgresql.org/download/)
2. Follow instructions to [Install PgVector](https://github.com/pgvector/pgvector#installation) in case you need to manually install it.
</TabItem>
</Tabs>
<h3>2. Create the Khoj database</h3>
<Tabs groupId="operating-systems" queryString="os">
<TabItem value="macos" label="MacOS">
```shell
createdb khoj -U postgres --password
```
</TabItem>
<TabItem value="windows" label="Windows">
```shell
createdb -U postgres khoj --password
```
</TabItem>
<TabItem value="linux" label="Linux">
```shell
sudo -u postgres createdb khoj --password
```
</TabItem>
</Tabs>
:::info[Postgres Env Config]
Make sure to update the `POSTGRES_HOST`, `POSTGRES_PORT`, `POSTGRES_USER`, `POSTGRES_DB` or `POSTGRES_PASSWORD` environment variables to match any customizations in your Postgres configuration.
:::
<h3>3. Install Khoj Server</h3>
<h3>1. Install Khoj Server</h3>
- Make sure [python](https://realpython.com/installing-python/) and [pip](https://pip.pypa.io/en/stable/installation/) are installed on your machine
- Check [llama-cpp-python setup](https://github.com/abetlen/llama-cpp-python?tab=readme-ov-file#supported-backends) if you hit any llama-cpp issues with the installation
@@ -127,12 +122,12 @@ Run the following command in your terminal to install the Khoj server.
<Tabs groupId="gpu" queryString="gpu">
<TabItem value="arm" label="ARM/M1+">
```shell
CMAKE_ARGS="-DGGML_METAL=on" python -m pip install khoj
CMAKE_ARGS="-DGGML_METAL=on" python -m pip install 'khoj[local]'
```
</TabItem>
<TabItem value="intel" label="Intel">
```shell
python -m pip install khoj
python -m pip install 'khoj[local]'
```
</TabItem>
</Tabs>
@@ -143,7 +138,7 @@ python -m pip install khoj
<TabItem value="cpu" label="CPU">
```shell
# Install Khoj
py -m pip install khoj
py -m pip install 'khoj[local]'
```
</TabItem>
<TabItem value="nvidia" label="NVIDIA (CUDA) GPU">
@@ -151,7 +146,7 @@ python -m pip install khoj
# 1. To use NVIDIA (CUDA) GPU
$env:CMAKE_ARGS = "-DGGML_CUDA=on"
# 2. Install Khoj
py -m pip install khoj
py -m pip install 'khoj[local]'
```
</TabItem>
<TabItem value="amd" label="AMD (ROCm) GPU">
@@ -159,7 +154,7 @@ python -m pip install khoj
# 1. To use AMD (ROCm) GPU
$env:CMAKE_ARGS = "-DGGML_HIPBLAS=on"
# 2. Install Khoj
py -m pip install khoj
py -m pip install 'khoj[local]'
```
</TabItem>
<TabItem value="vulkan" label="VULKAN GPU">
@@ -167,7 +162,7 @@ python -m pip install khoj
# 1. To use VULCAN GPU
$env:CMAKE_ARGS = "-DGGML_VULKAN=on"
# 2. Install Khoj
py -m pip install khoj
py -m pip install 'khoj[local]'
```
</TabItem>
</Tabs>
@@ -176,43 +171,43 @@ python -m pip install khoj
<Tabs groupId="gpu" queryString="gpu">
<TabItem value="cpu" label="CPU">
```shell
python -m pip install khoj
python -m pip install 'khoj[local]'
```
</TabItem>
<TabItem value="nvidia" label="NVIDIA (CUDA) GPU">
```shell
CMAKE_ARGS="-DGGML_CUDA=on" FORCE_CMAKE=1 python -m pip install khoj
CMAKE_ARGS="-DGGML_CUDA=on" FORCE_CMAKE=1 python -m pip install 'khoj[local]'
```
</TabItem>
<TabItem value="amd" label="AMD (ROCm) GPU">
```shell
CMAKE_ARGS="-DGGML_HIPBLAS=on" FORCE_CMAKE=1 python -m pip install khoj
CMAKE_ARGS="-DGGML_HIPBLAS=on" FORCE_CMAKE=1 python -m pip install 'khoj[local]'
```
</TabItem>
<TabItem value="vulkan" label="VULKAN GPU">
```shell
CMAKE_ARGS="-DGGML_VULKAN=on" FORCE_CMAKE=1 python -m pip install khoj
CMAKE_ARGS="-DGGML_VULKAN=on" FORCE_CMAKE=1 python -m pip install 'khoj[local]'
```
</TabItem>
</Tabs>
</TabItem>
</Tabs>
<h3>4. Start Khoj Server</h3>
<h3>2. Start Khoj Server</h3>
Run the following command from your terminal to start the Khoj service.
```shell
khoj --anonymous-mode
USE_EMBEDDED_DB="true" khoj --anonymous-mode
```
`--anonymous-mode` allows access to Khoj without requiring login. This is usually fine for local only, single user setups. If you need authentication follow the [authentication setup docs](/advanced/authentication).
<h4>First Run</h4>
On the first run of the above command, you will be prompted to:
1. Create an admin account with a email and secure password
1. Create an admin account with an email and secure password
2. Customize the chat models to enable
- Keep your [OpenAI](https://platform.openai.com/api-keys), [Anthropic](https://console.anthropic.com/account/keys), [Gemini](https://aistudio.google.com/app/apikey) API keys and [OpenAI](https://platform.openai.com/docs/models), [Anthropic](https://docs.anthropic.com/en/docs/about-claude/models#model-names), [Gemini](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models#gemini-models), [Offline](https://huggingface.co/models?pipeline_tag=text-generation&library=gguf) chat model names handy to set any of them up during first run.
3. Your setup is complete once you see `🌖 Khoj is ready to use` in the server logs on your terminal!
3. Your setup is complete once you see `🌖 Khoj is ready to engage` in the server logs on your terminal!
:::tip[Auto Start]
@@ -226,10 +221,6 @@ To start Khoj automatically in the background use [Task scheduler](https://www.w
You can now open the web app at http://localhost:42110 and start interacting!<br />
Nothing else is necessary, but you can customize your setup further by following the steps below.
:::info[First Message to Offline Chat Model]
The offline chat model gets downloaded when you first send a message to it. The download can take a few minutes! Subsequent messages should be faster.
:::
### Add Chat Models
<h4>Login to the Khoj Admin Panel</h4>
Go to http://localhost:42110/server/admin and login with the admin credentials you setup during installation.
@@ -238,9 +229,14 @@ Go to http://localhost:42110/server/admin and login with the admin credentials y
Ensure you are using **localhost, not 127.0.0.1**, to access the admin panel to avoid the CSRF error.
:::
:::info[CSRF Trusted Origin or Unset Cookie Error]
If using a load balancer/reverse_proxy in front of your Khoj server: Set the environment variable KHOJ_ALLOWED_DOMAIN=your-internal-ip-or-domain to avoid this error.
If unset, it defaults to KHOJ_DOMAIN.
:::
:::info[DISALLOWED HOST or Bad Request (400) Error]
You may hit this if you try access Khoj exposed on a custom domain (e.g. 192.168.12.3 or example.com) or over HTTP.
Set the environment variables KHOJ_DOMAIN=your-domain and KHOJ_NO_HTTPS=True if required to avoid this error.
Set the environment variables KHOJ_DOMAIN=your-external-ip-or-domain and KHOJ_NO_HTTPS=True if required to avoid this error.
:::
:::tip[Note]
@@ -257,12 +253,12 @@ Setup which chat model you'd want to use. Khoj supports local and online chat mo
Using Ollama? See the [Ollama Integration](/advanced/ollama) section for more custom setup instructions.
:::
1. Create a new [OpenAI processor conversation config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) in the server admin settings. This is kind of a misnomer, we know.
1. Create a new [AI Model Api](http://localhost:42110/server/admin/database/aimodelapi/add) in the server admin settings.
- Add your [OpenAI API key](https://platform.openai.com/api-keys)
- Give the configuration a friendly name like `OpenAI`
- (Optional) Set the API base URL. It is only relevant if you're using another OpenAI-compatible proxy server like [Ollama](/advanced/ollama) or [LMStudio](/advanced/lmstudio).<br />
![example configuration for openai processor](/img/example_openai_processor_config.png)
2. Create a new [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/add)
![example configuration for ai model api](/img/example_openai_processor_config.png)
2. Create a new [chat model](http://localhost:42110/server/admin/database/chatmodel/add)
- Set the `chat-model` field to an [OpenAI chat model](https://platform.openai.com/docs/models). Example: `gpt-4o`.
- Make sure to set the `model-type` field to `OpenAI`.
- If your model supports vision, set the `vision enabled` field to `true`. This is currently only supported for OpenAI models with vision capabilities.
@@ -270,22 +266,22 @@ Using Ollama? See the [Ollama Integration](/advanced/ollama) section for more cu
![example configuration for chat model options](/img/example_chatmodel_option.png)
</TabItem>
<TabItem value="anthropic" label="Anthropic">
1. Create a new [OpenAI processor conversation config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) in the server admin settings. This is kind of a misnomer, we know.
1. Create a new [AI Model API](http://localhost:42110/server/admin/database/aimodelapi/add) in the server admin settings.
- Add your [Anthropic API key](https://console.anthropic.com/account/keys)
- Give the configuration a friendly name like `Anthropic`. Do not configure the API base url.
2. Create a new [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/add)
2. Create a new [chat model](http://localhost:42110/server/admin/database/chatmodel/add)
- Set the `chat-model` field to an [Anthropic chat model](https://docs.anthropic.com/en/docs/about-claude/models#model-names). Example: `claude-3-5-sonnet-20240620`.
- Set the `model-type` field to `Anthropic`.
- Set the `Openai config` field to the OpenAI processor conversation config for Anthropic you created in step 1.
- Set the `ai model api` field to the Anthropic AI Model API you created in step 1.
</TabItem>
<TabItem value="gemini" label="Gemini">
1. Create a new [OpenAI processor conversation config](http://localhost:42110/server/admin/database/openaiprocessorconversationconfig/add) in the server admin settings. This is kind of a misnomer, we know.
1. Create a new [AI Model API](http://localhost:42110/server/admin/database/aimodelapi/add) in the server admin settings.
- Add your [Gemini API key](https://aistudio.google.com/app/apikey)
- Give the configuration a friendly name like `Gemini`. Do not configure the API base url.
2. Create a new [chat model options](http://localhost:42110/server/admin/database/chatmodeloptions/add)
- Set the `chat-model` field to a [Google Gemini chat model](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models#gemini-models). Example: `gemini-1.5-flash`.
- Set the `model-type` field to `Gemini`.
- Set the `Openai config` field to the OpenAI processor conversation config for Gemini you created in step 1.
2. Create a new [chat model](http://localhost:42110/server/admin/database/chatmodel/add)
- Set the `chat-model` field to a [Google Gemini chat model](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models#gemini-models). Example: `gemini-2.5-flash`.
- Set the `model-type` field to `Google`.
- Set the `ai model api` field to the Gemini AI Model API you created in step 1.
</TabItem>
<TabItem value="offline" label="Offline">
@@ -297,13 +293,14 @@ Offline chat stays completely private and can work without internet using any op
- A Nvidia, AMD GPU or a Mac M1+ machine would significantly speed up chat responses
:::
1. Get the name of your preferred chat model from [HuggingFace](https://huggingface.co/models?pipeline_tag=text-generation&library=gguf). *Most GGUF format chat models are supported*.
2. Open the [create chat model page](http://localhost:42110/server/admin/database/chatmodeloptions/add/) on the admin panel
3. Set the `chat-model` field to the name of your preferred chat model
- Make sure the `model-type` is set to `Offline`
4. Set the newly added chat model as your preferred model in your [User chat settings](http://localhost:42110/settings) and [Server chat settings](http://localhost:42110/server/admin/database/serverchatsettings/).
5. Restart the Khoj server and [start chatting](http://localhost:42110) with your new offline model!
</TabItem>
1. Install any Openai API compatible local ai model server like [llama-cpp-server](https://github.com/ggml-org/llama.cpp/tree/master/tools/server), Ollama, vLLM etc.
2. Add an [ai model api](http://localhost:42110/server/admin/database/aimodelapi/add/) on the admin panel
- Set the `api url` field to the url of your local ai model provider like `http://localhost:11434/v1/` for Ollama
3. Restart the Khoj server to load models available on your local ai model provider
- If that doesn't work, you'll need to manually add available [chat model](http://localhost:42110/server/admin/database/chatmodel/add) in the admin panel.
4. Set the newly added chat model as your preferred model in your [User chat settings](http://localhost:42110/settings)
5. [Start chatting](http://localhost:42110) with your local AI!
</TabItem>
</Tabs>
:::tip[Multiple Chat Models]

10
documentation/docs/miscellaneous/performance.md
View File

@@ -6,20 +6,22 @@ sidebar_position: 2
Here are some top-level performance metrics for Khoj. These are rough estimates and will vary based on your hardware and data.
:::info
These performance metrics were last evaluated in 2022.
:::
### Search performance
- Semantic search using the bi-encoder is fairly fast at \<100 ms across all content types
- Reranking using the cross-encoder is slower at \<2s on 15 results. Tweak `top_k` to tradeoff speed for accuracy of results
- Semantic search using the default embeddings model is fairly fast at \<100 ms across all content types
- Reranking using the cross-encoder model is slower at \<2s on 15 results. Tweak `top_k` to tradeoff speed for accuracy of results
- Filters in query (e.g. by file, word or date) usually add \<20ms to query latency
### Indexing performance
- Indexing is more strongly impacted by the size of the source data
- Indexing 100K+ line corpus of notes takes about 10 minutes
- Indexing 4000+ images takes about 15 minutes and more than 8Gb of RAM
- Note: *It should only take this long on the first run* as the index is incrementally updated
### Miscellaneous
- Testing done on a Mac M1 and a \>100K line corpus of notes
- Search, indexing on a GPU has not been tested yet

4
documentation/docs/miscellaneous/advanced.md → documentation/docs/miscellaneous/query-filters.md
View File

@@ -2,9 +2,7 @@
sidebar_position: 3
---
# Advanced Usage
## Query Filters
# Query Filters
Use structured query syntax to filter entries from your knowledge based used by search results or chat responses.

9
documentation/docs/miscellaneous/telemetry.md
View File

@@ -14,13 +14,6 @@ We don't send any personal information or any information from/about your conten
## Disable Telemetry
If you're self-hosting Khoj, you can opt out of telemetry at any time. To do so,
1. Open `~/.khoj/khoj.yml`
2. Add the following configuration:
```
app:
should-log-telemetry: false
```
3. Save the file and restart Khoj
If you're self-hosting Khoj, you can opt out of telemetry at any time by setting the `KHOJ_TELEMETRY_DISABLE` environment variable to `True` via shell or `docker-compose.yml`
If you have any questions or concerns, please reach out to us on [Discord](https://discord.gg/BDgyabRM6e).

39
documentation/docusaurus.config.js
View File

@@ -27,7 +27,11 @@ const config = {
projectName: 'khoj', // Usually your repo name.
onBrokenLinks: 'throw',
onBrokenMarkdownLinks: 'warn',
markdown: {
hooks: {
onBrokenMarkdownLinks: 'warn',
},
},
// Even if you don't use internationalization, you can use this field to set
// useful metadata like html lang. For example, if your site is Chinese, you
@@ -37,6 +41,9 @@ const config = {
locales: ['en'],
},
// Add a widget for Chatwoot for live chat if users need help
clientModules: [require.resolve('./src/components/ChatwootWidget.js')],
presets: [
[
'classic',
@@ -69,19 +76,17 @@ const config = {
}),
],
],
themeConfig:
/** @type {import('@docusaurus/preset-classic').ThemeConfig} */
({
image: 'img/khoj-logo-sideways-500.png',
image: 'img/khoj_documentation.png',
metadata: [
{name: 'og:title', content: 'Khoj Documentation'},
{name: 'og:title', content: 'Docs'},
{name: 'og:type', content: 'website'},
{name: 'og:site_name', content: 'Khoj Documentation'},
{name: 'og:description', content: 'Quickly get started with using or self-hosting Khoj'},
{name: 'og:image', content: 'https://assets.khoj.dev/link_preview_docs.png'},
{name: 'og:url', content: 'https://docs.khoj.dev'},
{name: 'keywords', content: 'khoj, khoj ai, chatgpt, open ai, open source, productivity'}
{name: 'keywords', content: 'khoj, khoj ai, chatgpt, open source ai, open source, transparent, accessible, trustworthy, hackable, index notes, rag, productivity'}
],
navbar: {
title: 'Khoj',
@@ -92,23 +97,31 @@ const config = {
items: [
{
href: 'https://github.com/khoj-ai/khoj',
label: 'GitHub',
position: 'right',
className: 'header-github-link',
title: 'Codebase',
'aria-label': 'GitHub repository',
},
{
href: 'https://app.khoj.dev/login',
label: 'Cloud',
href: 'https://app.khoj.dev',
position: 'right',
className: 'header-cloud-link',
title: 'Khoj Cloud',
'aria-label': 'Khoj Cloud',
},
{
href: 'https://discord.gg/BDgyabRM6e',
label: 'Discord',
position: 'right',
className: 'header-discord-link',
title: 'Community',
'aria-label': 'Discord community',
},
{
href: 'https://blog.khoj.dev',
label: 'Blog',
position: 'right',
className: 'header-blog-link',
title: 'Blog',
'aria-label': 'Khoj Blog',
},
],
},
@@ -178,14 +191,14 @@ const config = {
},
{
label: 'Khoj Cloud',
href: 'https://app.khoj.dev/login',
href: 'https://app.khoj.dev',
},
{
label: 'GitHub',
href: 'https://github.com/khoj-ai/khoj',
},
{
label: 'Website',
label: 'Khoj Inc.',
href: 'https://khoj.dev',
},
],

20
documentation/package.json
View File

@@ -14,18 +14,19 @@
"write-heading-ids": "docusaurus write-heading-ids"
},
"dependencies": {
"@docusaurus/core": "^3.2.1",
"@docusaurus/plugin-sitemap": "^3.2.1",
"@docusaurus/preset-classic": "^3.2.1",
"@docusaurus/core": "^3.9.2",
"@docusaurus/plugin-sitemap": "^3.9.2",
"@docusaurus/preset-classic": "^3.9.2",
"@mdx-js/react": "^3.0.0",
"clsx": "^2.0.0",
"prism-react-renderer": "^2.3.0",
"react": "^18.0.0",
"react-dom": "^18.0.0"
"react": "^19.1.0",
"react-dom": "^19.1.0",
"webpack-dev-server": "^5.2.1"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "^3.2.1",
"@docusaurus/types": "^3.2.1"
"@docusaurus/module-type-aliases": "^3.9.2",
"@docusaurus/types": "^3.9.2"
},
"browserslist": {
"production": [
@@ -41,5 +42,10 @@
},
"engines": {
"node": ">=18.0"
},
"resolutions": {
"webpack-dev-server": "^5.2.1",
"serialize-javascript": "^7.0.3",
"picomatch": ">=2.3.2"
}
}

19
documentation/src/components/ChatwootWidget.js Normal file
View File

@@ -0,0 +1,19 @@
import ExecutionEnvironment from '@docusaurus/ExecutionEnvironment';
// Only execute on client-side
if (ExecutionEnvironment.canUseDOM) {
(function (d, t) {
var BASE_URL = "https://app.chatwoot.com";
var g = d.createElement(t), s = d.getElementsByTagName(t)[0];
g.src = BASE_URL + "/packs/js/sdk.js";
g.defer = true;
g.async = true;
s.parentNode.insertBefore(g, s);
g.onload = function () {
window.chatwootSDK.run({
websiteToken: 'cFxvnLSjfE2UF4UUiPCA5NsF',
baseUrl: BASE_URL
})
}
})(document, 'script');
}

76
documentation/src/css/custom.css
View File

@@ -8,13 +8,13 @@
/* You can override the default Infima variables here. */
:root {
--ifm-color-primary: #fcc50b;
--ifm-color-primary-dark: #fcc50b;
--ifm-color-primary-darker: #fcc50b;
--ifm-color-primary-darkest: #fcc50b;
--ifm-color-primary-light: #fcc50b;
--ifm-color-primary-lighter: #fcc50b;
--ifm-color-primary-lightest: #fcc50b;
--ifm-color-primary: #FFA07A;
--ifm-color-primary-dark: #FFA07A;
--ifm-color-primary-darker: #FFA07A;
--ifm-color-primary-darkest: #FFA07A;
--ifm-color-primary-light: #FFA07A;
--ifm-color-primary-lighter: #FFA07A;
--ifm-color-primary-lightest: #FFA07A;
--ifm-code-font-size: 95%;
--ifm-heading-font-family: 'Source Sans 3', sans-serif;
--docusaurus-highlighted-code-line-bg: rgba(0, 0, 0, 0.1);
@@ -22,16 +22,64 @@
/* For readability concerns, you should choose a lighter palette in dark mode. */
[data-theme='dark'] {
--ifm-color-primary: #fcc50b;
--ifm-color-primary-dark: #fcc50b;
--ifm-color-primary-darker: #fcc50b;
--ifm-color-primary-darkest: #fcc50b;
--ifm-color-primary-light: #fcc50b;
--ifm-color-primary-lighter: #fcc50b;
--ifm-color-primary-lightest: #fcc50b;
--ifm-color-primary: #FFA07A;
--ifm-color-primary-dark: #FFA07A;
--ifm-color-primary-darker: #FFA07A;
--ifm-color-primary-darkest: #FFA07A;
--ifm-color-primary-light: #FFA07A;
--ifm-color-primary-lighter: #FFA07A;
--ifm-color-primary-lightest: #FFA07A;
--docusaurus-highlighted-code-line-bg: rgba(0, 0, 0, 0.3);
}
body {
font-family: 'Source Sans 3', sans-serif;
}
.header-github-link::before {
content: '';
width: 24px;
height: 24px;
display: flex;
background-color: var(--ifm-navbar-link-color);
mask-image: url("data:image/svg+xml,%3Csvg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12'/%3E%3C/svg%3E");
transition: background-color var(--ifm-transition-fast) var(--ifm-transition-timing-default);
}
.header-blog-link::before {
content: '';
width: 24px;
height: 24px;
display: flex;
background-color: var(--ifm-navbar-link-color);
mask-image: url("data:image/svg+xml,%3Csvg viewBox='0 0 512 512' xmlns='http://www.w3.org/2000/svg'%3E%3C!--!Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free Copyright 2024 Fonticons, Inc.--%3E%3Cpath d='M278.5 215.6L23 471c-9.4 9.4-9.4 24.6 0 33.9s24.6 9.4 33.9 0l57-57 68 0c49.7 0 97.9-14.4 139-41c11.1-7.2 5.5-23-7.8-23c-5.1 0-9.2-4.1-9.2-9.2c0-4.1 2.7-7.6 6.5-8.8l81-24.3c2.5-.8 4.8-2.1 6.7-4l22.4-22.4c10.1-10.1 2.9-27.3-11.3-27.3l-32.2 0c-5.1 0-9.2-4.1-9.2-9.2c0-4.1 2.7-7.6 6.5-8.8l112-33.6c4-1.2 7.4-3.9 9.3-7.7C506.4 207.6 512 184.1 512 160c0-41-16.3-80.3-45.3-109.3l-5.5-5.5C432.3 16.3 393 0 352 0s-80.3 16.3-109.3 45.3L139 149C91 197 64 262.1 64 330l0 55.3L253.6 195.8c6.2-6.2 16.4-6.2 22.6 0c5.4 5.4 6.1 13.6 2.2 19.8z'/%3E%3C/svg%3E");
transition: background-color var(--ifm-transition-fast) var(--ifm-transition-timing-default);
}
.header-cloud-link::before {
content: '';
width: 24px;
height: 24px;
display: flex;
background-color: var(--ifm-navbar-link-color);
mask-image: url("data:image/svg+xml,%3Csvg viewBox='0 0 512 512' xmlns='http://www.w3.org/2000/svg'%3E%3C!--!Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free Copyright 2024 Fonticons, Inc.--%3E%3Cpath d='M0 224c0 53 43 96 96 96l47.2 0L290 202.5c17.6-14.1 42.6-14 60.2 .2s22.8 38.6 12.8 58.8L333.7 320l18.3 0 64 0c53 0 96-43 96-96s-43-96-96-96c-.5 0-1.1 0-1.6 0c1.1-5.2 1.6-10.5 1.6-16c0-44.2-35.8-80-80-80c-24.3 0-46.1 10.9-60.8 28C256.5 24.3 219.1 0 176 0C114.1 0 64 50.1 64 112c0 7.1 .7 14.1 1.9 20.8C27.6 145.4 0 181.5 0 224zm330.1 3.6c-5.8-4.7-14.2-4.7-20.1-.1l-160 128c-5.3 4.2-7.4 11.4-5.1 17.8s8.3 10.7 15.1 10.7l70.1 0L177.7 488.8c-3.4 6.7-1.6 14.9 4.3 19.6s14.2 4.7 20.1 .1l160-128c5.3-4.2 7.4-11.4 5.1-17.8s-8.3-10.7-15.1-10.7l-70.1 0 52.4-104.8c3.4-6.7 1.6-14.9-4.2-19.6z'/%3E%3C/svg%3E");
transition: background-color var(--ifm-transition-fast) var(--ifm-transition-timing-default);
}
.header-discord-link::before {
content: '';
width: 30px;
height: 24px;
display: flex;
background-color: var(--ifm-navbar-link-color);
mask-image: url("data:image/svg+xml,%3Csvg viewBox='0 0 640 512' xmlns='http://www.w3.org/2000/svg'%3E%3C!--!Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free Copyright 2024 Fonticons, Inc.--%3E%3Cpath d='M524.5 69.8a1.5 1.5 0 0 0 -.8-.7A485.1 485.1 0 0 0 404.1 32a1.8 1.8 0 0 0 -1.9 .9 337.5 337.5 0 0 0 -14.9 30.6 447.8 447.8 0 0 0 -134.4 0 309.5 309.5 0 0 0 -15.1-30.6 1.9 1.9 0 0 0 -1.9-.9A483.7 483.7 0 0 0 116.1 69.1a1.7 1.7 0 0 0 -.8 .7C39.1 183.7 18.2 294.7 28.4 404.4a2 2 0 0 0 .8 1.4A487.7 487.7 0 0 0 176 479.9a1.9 1.9 0 0 0 2.1-.7A348.2 348.2 0 0 0 208.1 430.4a1.9 1.9 0 0 0 -1-2.6 321.2 321.2 0 0 1 -45.9-21.9 1.9 1.9 0 0 1 -.2-3.1c3.1-2.3 6.2-4.7 9.1-7.1a1.8 1.8 0 0 1 1.9-.3c96.2 43.9 200.4 43.9 295.5 0a1.8 1.8 0 0 1 1.9 .2c2.9 2.4 6 4.9 9.1 7.2a1.9 1.9 0 0 1 -.2 3.1 301.4 301.4 0 0 1 -45.9 21.8 1.9 1.9 0 0 0 -1 2.6 391.1 391.1 0 0 0 30 48.8 1.9 1.9 0 0 0 2.1 .7A486 486 0 0 0 610.7 405.7a1.9 1.9 0 0 0 .8-1.4C623.7 277.6 590.9 167.5 524.5 69.8zM222.5 337.6c-29 0-52.8-26.6-52.8-59.2S193.1 219.1 222.5 219.1c29.7 0 53.3 26.8 52.8 59.2C275.3 311 251.9 337.6 222.5 337.6zm195.4 0c-29 0-52.8-26.6-52.8-59.2S388.4 219.1 417.9 219.1c29.7 0 53.3 26.8 52.8 59.2C470.7 311 447.5 337.6 417.9 337.6z'/%3E%3C/svg%3E");
transition: background-color var(--ifm-transition-fast) var(--ifm-transition-timing-default);
}
.header-github-link:hover::before,
.header-cloud-link:hover::before,
.header-discord-link:hover::before,
.header-blog-link:hover::before {
background-color: var(--ifm-navbar-link-hover-color);
transform: scale(1.1);
}

6364
documentation/yarn.lock
View File

File diff suppressed because it is too large Load Diff

15
gunicorn-config.py
View File

@@ -1,10 +1,17 @@
import multiprocessing
import os
bind = "0.0.0.0:42110"
workers = 2
# Worker Configuration
workers = int(os.environ.get("GUNICORN_WORKERS", 6))
worker_class = "uvicorn.workers.UvicornWorker"
timeout = 120
keep_alive = 60
# Worker Timeout Configuration
timeout = int(os.environ.get("GUNICORN_TIMEOUT", 180))
graceful_timeout = int(os.environ.get("GUNICORN_GRACEFUL_TIMEOUT", 90))
keep_alive = int(os.environ.get("GUNICORN_KEEP_ALIVE", 60))
# Logging Configuration
accesslog = "-"
errorlog = "-"
loglevel = "debug"

2
manifest.json
View File

@@ -1,7 +1,7 @@
{
"id": "khoj",
"name": "Khoj",
"version": "1.26.2",
"version": "2.0.0-beta.28",
"minAppVersion": "0.15.0",
"description": "Your Second Brain",
"author": "Khoj Inc.",

50
prod.Dockerfile
View File

@@ -1,5 +1,5 @@
# syntax=docker/dockerfile:1
FROM ubuntu:jammy
FROM ubuntu:jammy AS base
LABEL homepage="https://khoj.dev"
LABEL repository="https://github.com/khoj-ai/khoj"
LABEL org.opencontainers.image.source="https://github.com/khoj-ai/khoj"
@@ -13,36 +13,52 @@ RUN apt update -y && apt -y install \
libsm6 \
libxext6 \
swig \
curl && \
# Required by Next.js Web app
curl -sL https://deb.nodesource.com/setup_20.x | bash - && \
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \
apt update -y && apt -y --no-install-recommends install nodejs yarn && \
curl \
# Required by llama-cpp-python pre-built wheels. See #1628
musl-dev && \
ln -s /usr/lib/x86_64-linux-musl/libc.so /lib/libc.musl-x86_64.so.1 && \
# Clean up
apt clean && rm -rf /var/lib/apt/lists/*
# Install Application
# Build Server
FROM base AS server-deps
WORKDIR /app
COPY pyproject.toml .
COPY README.md .
ARG VERSION=0.0.0
# use the pre-built llama-cpp-python, torch cpu wheel
ENV PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cpu https://abetlen.github.io/llama-cpp-python/whl/cpu"
# avoid downloading unused cuda specific python packages
ENV CUDA_VISIBLE_DEVICES=""
RUN sed -i "s/dynamic = \\[\"version\"\\]/version = \"$VERSION\"/" pyproject.toml && \
pip install --no-cache-dir -e .[prod]
# Copy Source Code
COPY . .
# Set the PYTHONPATH environment variable in order for it to find the Django app.
ENV PYTHONPATH=/app/src:$PYTHONPATH
# Go to the directory src/interface/web and export the built Next.js assets
# Build Web App
FROM oven/bun:1-alpine AS web-app
# Set build optimization env vars
ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1
WORKDIR /app/src/interface/web
RUN bash -c "yarn install --frozen-lockfile --verbose && yarn ciexport && yarn cache clean"
# Install dependencies first (cache layer)
COPY src/interface/web/package.json src/interface/web/bun.lock ./
RUN bun install --frozen-lockfile
# Copy source and build
COPY src/interface/web/. ./
RUN bun run build
# Merge the Server and Web App into a Single Image
FROM base
ENV PYTHONPATH=/app/src:$PYTHONPATH
WORKDIR /app
COPY --from=server-deps /usr/local/lib/python3.10/dist-packages /usr/local/lib/python3.10/dist-packages
COPY --from=server-deps /usr/local/bin /usr/local/bin
COPY --from=web-app /app/src/interface/web/out ./src/khoj/interface/built
COPY . .
RUN cd src && python3 khoj/manage.py collectstatic --noinput
# Run the Application
# There are more arguments required for the application to run,
# but these should be passed in through the docker-compose.yml file.
# but those should be passed in through the docker-compose.yml file.
ARG PORT
EXPOSE ${PORT}
ENTRYPOINT ["gunicorn", "-c", "gunicorn-config.py", "src.khoj.main:app"]

104
pyproject.toml
View File

@@ -7,7 +7,7 @@ name = "khoj"
description = "Your Second Brain"
readme = "README.md"
license = "AGPL-3.0-or-later"
requires-python = ">=3.10"
requires-python = ">=3.10, <3.13"
authors = [
{ name = "Debanjum Singh Solanky, Saba Imran" },
]
@@ -40,54 +40,61 @@ dependencies = [
"dateparser >= 1.1.1",
"defusedxml == 0.7.1",
"fastapi >= 0.110.0",
"python-multipart >= 0.0.7",
"jinja2 == 3.1.4",
"openai >= 1.0.0",
"python-multipart >= 0.0.22",
"jinja2 == 3.1.6",
"openai >= 2.0.0, < 3.0.0",
"tiktoken >= 0.3.2",
"tenacity >= 8.2.2",
"tenacity >= 9.0.0",
"magika ~= 0.5.1",
"pillow ~= 10.0.0",
"pillow >= 12.1.1",
"pydantic[email] >= 2.0.0",
"pyyaml ~= 6.0",
"rich >= 13.3.1",
"click < 8.2.2",
"schedule == 1.1.0",
"sentence-transformers == 3.0.1",
"sentence-transformers == 3.4.1",
"einops == 0.8.0",
"transformers >= 4.28.0",
"torch == 2.2.2",
"uvicorn == 0.30.6",
"aiohttp ~= 3.9.0",
"langchain == 0.2.5",
"langchain-community == 0.2.5",
"requests >= 2.26.0",
"tenacity == 8.3.0",
"anyio == 3.7.1",
"pymupdf >= 1.23.5",
"django == 5.0.9",
"authlib == 1.2.1",
"llama-cpp-python == 0.2.88",
"transformers >= 4.53.0",
"torch == 2.6.0",
"uvicorn >= 0.31.1",
"aiohttp ~= 3.13.0",
"langchain-text-splitters == 0.3.11",
"langchain-community == 0.3.31",
"requests >= 2.33.0",
"anyio ~= 4.8.0",
"pymupdf == 1.24.11",
"django == 5.1.15",
"django-unfold == 0.42.0",
"authlib == 1.6.9",
"itsdangerous == 2.1.2",
"httpx == 0.25.0",
"httpx == 0.28.1",
"pgvector == 0.2.4",
"psycopg2-binary == 2.9.9",
"lxml == 4.9.3",
"tzdata == 2023.3",
"rapidocr-onnxruntime == 1.3.24",
"rapidocr-onnxruntime == 1.4.4",
"openai-whisper >= 20231117",
"django-phonenumber-field == 7.3.0",
"phonenumbers == 8.13.27",
"markdownify ~= 0.11.6",
"markdownify ~= 0.14.1",
"markdown-it-py ~= 3.0.0",
"websockets == 12.0",
"websockets == 13.0",
"psutil >= 5.8.0",
"huggingface-hub >= 0.22.2",
"huggingface-hub[hf_xet] >= 0.22.2",
"apscheduler ~= 3.10.0",
"pytz ~= 2024.1",
"cron-descriptor == 1.4.3",
"django_apscheduler == 0.6.2",
"anthropic == 0.26.1",
"django_apscheduler == 0.7.0",
"anthropic == 0.75.0",
"docx2txt == 0.8",
"google-generativeai == 0.7.2"
"google-genai == 1.52.0",
"google-auth ~= 2.23.3",
"pyjson5 == 1.6.7",
"resend == 1.2.0",
"email-validator == 2.2.0",
"e2b-code-interpreter ~= 1.0.0",
"mcp >= 1.23.0",
"pyasn1>=0.6.3",
]
dynamic = ["version"]
@@ -102,14 +109,15 @@ khoj = "khoj.main:run"
[project.optional-dependencies]
prod = [
"gunicorn == 22.0.0",
"google-auth == 2.23.3",
"stripe == 7.3.0",
"twilio == 8.11",
"boto3 >= 1.34.57",
"resend == 1.0.1",
]
local = [
"pgserver == 0.1.4",
]
dev = [
"khoj[prod]",
"khoj[prod,local]",
"pytest >= 7.1.2",
"pytest-xdist[psutil]",
"pytest-django == 4.5.2",
@@ -117,8 +125,11 @@ dev = [
"freezegun >= 1.2.0",
"factory-boy >= 3.2.1",
"mypy >= 1.0.1",
"black >= 23.1.0",
"ruff >= 0.12.0",
"pre-commit >= 3.0.4",
"gitpython ~= 3.1.43",
"datasets",
"pandas",
]
[tool.hatch.version]
@@ -141,11 +152,34 @@ non_interactive = true
show_error_codes = true
warn_unused_ignores = false
[tool.black]
[tool.ruff]
line-length = 120
[tool.isort]
profile = "black"
[tool.ruff.lint]
select = ["E", "F", "I"] # Enable error, warning, and import checks
ignore = [
"E501", # Ignore line length
"F405", # Ignore name not defined (e.g., from imports)
"E402", # Ignore module level import not at top of file
]
unfixable = ["F841"] # Don't auto-remove unused variables
exclude = [ "tests/*.py" ]
[tool.ruff.lint.per-file-ignores]
"src/khoj/main.py" = [
"I001", # Ignore Import order
"I002", # Ignore Import not at top of file
"E402", # Ignore module level import not at top of file
]
[tool.ruff.format]
quote-style = "double"
indent-style = "space"
[tool.ruff.lint.isort]
known-first-party = ["khoj"]
[tool.uv]
build-constraint-dependencies = ["setuptools>=61.2,<82"]
[tool.pytest.ini_options]
addopts = "--strict-markers"

1
pytest.ini
View File

@@ -2,5 +2,6 @@
DJANGO_SETTINGS_MODULE = khoj.app.settings
pythonpath = . src
testpaths = tests
addopts = --reuse-db
markers =
chatquality: marks tests as chatquality (deselect with '-m "not chatquality"')

83
scripts/bump_version.sh
View File

@@ -2,9 +2,80 @@
project_root=$PWD
while getopts 'nc:t:' opt;
while getopts 'nc:t:p:' opt;
do
case "${opt}" in
p)
# Create pre-release SemVer version. Options: alpha, beta, rc
prerelease_type=$OPTARG
# Get the current version from web package.json to determine base version
cd $project_root/src/interface/web
current_base_version=$(grep '"version":' package.json | awk -F '"' '{print $4}')
# Extract base version (remove any existing pre-release suffix)
base_version=$(echo $current_base_version | sed 's/-.*$//')
# If current version is already 2.x.x, increment the pre-release number
if [[ $current_base_version == *"-$prerelease_type"* ]]; then
# Extract current pre-release number and increment
current_num=$(echo $current_base_version | sed "s/.*-$prerelease_type\.//" | sed 's/[^0-9]*$//')
next_num=$((current_num + 1))
current_version="$base_version-$prerelease_type.$next_num"
else
# If base version is 1.x.x, bump to 2.0.0-prerelease.1
if [[ $base_version == 1.* ]]; then
current_version="2.0.0-$prerelease_type.1"
else
# Otherwise add pre-release to current base version
current_version="$base_version-$prerelease_type.1"
fi
fi
# Bump Web app to pre-release version
cd $project_root/src/interface/web
yarn version --new-version $current_version --no-git-tag-version
# Bump Desktop app to pre-release version
cd $project_root/src/interface/desktop
yarn version --new-version $current_version --no-git-tag-version
# Bump Obsidian plugin to pre-release version
cd $project_root/src/interface/obsidian
yarn build # verify build before bumping version
yarn version --new-version $current_version --no-git-tag-version
# append current version, min Obsidian app version from manifest to versions json
cp $project_root/versions.json .
yarn run version # run Obsidian version script
# Bump Emacs package to pre-release version
cd ../emacs
sed -E -i.bak "s/^;; Version: (.*)/;; Version: $current_version/" khoj.el
git add khoj.el
rm *.bak
# Copy current obsidian versioned files to project root
cd $project_root
cp src/interface/obsidian/versions.json .
cp src/interface/obsidian/manifest.json .
# Run pre-commit validation to fix jsons
pre-commit run --hook-stage manual --all
# Commit changes and tag commit for pre-release
git add \
$project_root/src/interface/web/package.json \
$project_root/src/interface/desktop/package.json \
$project_root/src/interface/obsidian/package.json \
$project_root/src/interface/obsidian/yarn.lock \
$project_root/src/interface/obsidian/manifest.json \
$project_root/src/interface/obsidian/versions.json \
$project_root/src/interface/emacs/khoj.el \
$project_root/manifest.json \
$project_root/versions.json
git commit -m "Release Khoj version $current_version"
git tag $current_version
;;
t)
# Get version type to bump. Options: major, minor, patch
version_type=$OPTARG
@@ -54,7 +125,7 @@ do
$project_root/manifest.json \
$project_root/versions.json
git commit -m "Release Khoj version $current_version"
git tag $current_version master
git tag $current_version
;;
c)
# Get current project version
@@ -101,7 +172,7 @@ do
$project_root/manifest.json \
$project_root/versions.json
git commit -m "Release Khoj version $current_version"
git tag $current_version master
git tag $current_version
;;
n)
# Induce hatch to compute next version number
@@ -144,7 +215,11 @@ do
git commit -m "Bump Khoj to pre-release version $next_version"
;;
?)
echo -e "Invalid command option.\nUsage: $(basename $0) [-t] [-c] [-n]"
echo -e "Invalid command option.\nUsage: $(basename $0) [-t type] [-c version] [-p prerelease] [-n]"
echo -e " -t: Bump version by type (major, minor, patch)"
echo -e " -c: Set specific version"
echo -e " -p: Create pre-release version (alpha, beta, rc)"
echo -e " -n: Compute and set next version using hatch"
exit 1
;;
esac

85
scripts/dev_setup.sh
View File

@@ -2,30 +2,79 @@
# ---
PROJECT_ROOT=$(git rev-parse --show-toplevel)
# Install Web App
# ---
echo "Installing Web App..."
cd $PROJECT_ROOT/src/interface/web
yarn install
# Default to minimal installation unless --full flag passed
INSTALL_FULL=false
DEVCONTAINER=false
for arg in "$@"
do
if [ "$arg" = "--full" ]
then
INSTALL_FULL=true
fi
if [ "$arg" = "--devcontainer" ]
then
DEVCONTAINER=true
fi
done
if [ "$DEVCONTAINER" = true ]; then
echo "Dev container setup - using pre-installed dependencies..."
cd "$PROJECT_ROOT"
# Use devcontainer launch.json
mkdir -p .vscode && cp .devcontainer/launch.json .vscode/launch.json
# Install Server App using pre-installed dependencies
echo "Setup Server App with UV. Use pre-installed dependencies in $UV_PROJECT_ENVIRONMENT."
sed -i "s/dynamic = \\[\"version\"\\]/version = \"$VERSION\"/" pyproject.toml
cp /opt/uv.lock.linux uv.lock
uv sync --all-extras
# Install Web App using cached dependencies
echo "Setup Web App with Bun. Use pre-installed dependencies in /opt/khoj_web."
cd "$PROJECT_ROOT/src/interface/web"
ln -sf /opt/khoj_web/node_modules node_modules
bun install && bun run ciexport
else
# Standard setup
echo "Installing Server App..."
cd "$PROJECT_ROOT"
if command -v uv &> /dev/null
then
uv venv
uv sync --all-extras
else
python3 -m venv .venv && . .venv/bin/activate
python3 -m pip install -e '.[dev]'
fi
echo "Installing Web App..."
cd "$PROJECT_ROOT/src/interface/web"
if command -v bun &> /dev/null
then
echo "using Bun."
bun install && bun run export
else
echo "using Yarn."
yarn install && yarn export
fi
fi
# Install Obsidian App
# ---
echo "Installing Obsidian App..."
cd $PROJECT_ROOT/src/interface/obsidian
yarn install
if [ "$INSTALL_FULL" = true ] ; then
echo "Installing Obsidian App..."
cd $PROJECT_ROOT/src/interface/obsidian
yarn install
fi
# Install Desktop App
# ---
echo "Installing Desktop App..."
cd $PROJECT_ROOT/src/interface/desktop
yarn install
# Install Server App
# ---
echo "Installing Server App..."
cd $PROJECT_ROOT
# pip install --user pipenv && pipenv install -e '.[dev]' --skip-lock && pipenv shell
python3 -m venv .venv && pip install -e '.[dev]' && . .venv/bin/activate
if [ "$INSTALL_FULL" = true ] ; then
echo "Installing Desktop App..."
cd $PROJECT_ROOT/src/interface/desktop
yarn install
fi
# Install pre-commit hooks
# ----

211
src/interface/android/app/build.gradle Normal file
View File

@@ -0,0 +1,211 @@
/*
* Copyright 2019 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import groovy.xml.MarkupBuilder
plugins {
id 'com.android.application'
}
def twaManifest = [
applicationId: 'dev.khoj.app',
hostName: 'app.khoj.dev', // The domain being opened in the TWA.
launchUrl: '/', // The start path for the TWA. Must be relative to the domain.
name: 'Khoj AI', // The application name.
launcherName: 'Khoj', // The name shown on the Android Launcher.
themeColor: '#FFFFFF', // The color used for the status bar.
themeColorDark: '#000000', // The color used for the dark status bar.
navigationColor: '#000000', // The color used for the navigation bar.
navigationColorDark: '#000000', // The color used for the dark navbar.
navigationDividerColor: '#000000', // The navbar divider color.
navigationDividerColorDark: '#000000', // The dark navbar divider color.
backgroundColor: '#FFFFFF', // The color used for the splash screen background.
enableNotifications: true, // Set to true to enable notification delegation.
// Every shortcut must include the following fields:
// - name: String that will show up in the shortcut.
// - short_name: Shorter string used if |name| is too long.
// - url: Absolute path of the URL to launch the app with (e.g '/create').
// - icon: Name of the resource in the drawable folder to use as an icon.
shortcuts: [],
// The duration of fade out animation in milliseconds to be played when removing splash screen.
splashScreenFadeOutDuration: 300,
generatorApp: 'bubblewrap-cli', // Application that generated the Android Project
// The fallback strategy for when Trusted Web Activity is not available. Possible values are
// 'customtabs' and 'webview'.
fallbackType: 'customtabs',
enableSiteSettingsShortcut: 'true',
orientation: 'natural',
]
android {
compileSdkVersion 35
namespace "dev.khoj.app"
defaultConfig {
applicationId "dev.khoj.app"
minSdkVersion 19
targetSdkVersion 35
versionCode 4
versionName "4"
// The name for the application
resValue "string", "appName", twaManifest.name
// The name for the application on the Android Launcher
resValue "string", "launcherName", twaManifest.launcherName
// The URL that will be used when launching the TWA from the Android Launcher
def launchUrl = "https://" + twaManifest.hostName + twaManifest.launchUrl
resValue "string", "launchUrl", launchUrl
// The URL the Web Manifest for the Progressive Web App that the TWA points to. This
// is used by Chrome OS and Meta Quest to open the Web version of the PWA instead of
// the TWA, as it will probably give a better user experience for non-mobile devices.
resValue "string", "webManifestUrl", 'https://app.khoj.dev/static/khoj.webmanifest'
// This is used by Meta Quest.
resValue "string", "fullScopeUrl", 'https://app.khoj.dev/'
// The hostname is used when building the intent-filter, so the TWA is able to
// handle Intents to open host url of the application.
resValue "string", "hostName", twaManifest.hostName
// This attribute sets the status bar color for the TWA. It can be either set here or in
// `res/values/colors.xml`. Setting in both places is an error and the app will not
// compile. If not set, the status bar color defaults to #FFFFFF - white.
resValue "color", "colorPrimary", twaManifest.themeColor
// This attribute sets the dark status bar color for the TWA. It can be either set here or in
// `res/values/colors.xml`. Setting in both places is an error and the app will not
// compile. If not set, the status bar color defaults to #000000 - white.
resValue "color", "colorPrimaryDark", twaManifest.themeColorDark
// This attribute sets the navigation bar color for the TWA. It can be either set here or
// in `res/values/colors.xml`. Setting in both places is an error and the app will not
// compile. If not set, the navigation bar color defaults to #FFFFFF - white.
resValue "color", "navigationColor", twaManifest.navigationColor
// This attribute sets the dark navigation bar color for the TWA. It can be either set here
// or in `res/values/colors.xml`. Setting in both places is an error and the app will not
// compile. If not set, the navigation bar color defaults to #000000 - black.
resValue "color", "navigationColorDark", twaManifest.navigationColorDark
// This attribute sets the navbar divider color for the TWA. It can be either
// set here or in `res/values/colors.xml`. Setting in both places is an error and the app
// will not compile. If not set, the divider color defaults to #00000000 - transparent.
resValue "color", "navigationDividerColor", twaManifest.navigationDividerColor
// This attribute sets the dark navbar divider color for the TWA. It can be either
// set here or in `res/values/colors.xml`. Setting in both places is an error and the
//app will not compile. If not set, the divider color defaults to #000000 - black.
resValue "color", "navigationDividerColorDark", twaManifest.navigationDividerColorDark
// Sets the color for the background used for the splash screen when launching the
// Trusted Web Activity.
resValue "color", "backgroundColor", twaManifest.backgroundColor
// Defines a provider authority for the Splash Screen
resValue "string", "providerAuthority", twaManifest.applicationId + '.fileprovider'
// The enableNotification resource is used to enable or disable the
// TrustedWebActivityService, by changing the android:enabled and android:exported
// attributes
resValue "bool", "enableNotification", twaManifest.enableNotifications.toString()
twaManifest.shortcuts.eachWithIndex { shortcut, index ->
resValue "string", "shortcut_name_$index", "$shortcut.name"
resValue "string", "shortcut_short_name_$index", "$shortcut.short_name"
}
// The splashScreenFadeOutDuration resource is used to set the duration of fade out animation in milliseconds
// to be played when removing splash screen. The default is 0 (no animation).
resValue "integer", "splashScreenFadeOutDuration", twaManifest.splashScreenFadeOutDuration.toString()
resValue "string", "generatorApp", twaManifest.generatorApp
resValue "string", "fallbackType", twaManifest.fallbackType
resValue "bool", "enableSiteSettingsShortcut", twaManifest.enableSiteSettingsShortcut
resValue "string", "orientation", twaManifest.orientation
}
buildTypes {
release {
minifyEnabled true
}
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
lintOptions {
checkReleaseBuilds false
}
}
task generateShorcutsFile {
assert twaManifest.shortcuts.size() < 5, "You can have at most 4 shortcuts."
twaManifest.shortcuts.eachWithIndex { s, i ->
assert s.name != null, 'Missing `name` in shortcut #' + i
assert s.short_name != null, 'Missing `short_name` in shortcut #' + i
assert s.url != null, 'Missing `icon` in shortcut #' + i
assert s.icon != null, 'Missing `url` in shortcut #' + i
}
def shortcutsFile = new File("$projectDir/src/main/res/xml", "shortcuts.xml")
def xmlWriter = new StringWriter()
def xmlMarkup = new MarkupBuilder(new IndentPrinter(xmlWriter, " ", true))
xmlMarkup
.'shortcuts'('xmlns:android': 'http://schemas.android.com/apk/res/android') {
twaManifest.shortcuts.eachWithIndex { s, i ->
'shortcut'(
'android:shortcutId': 'shortcut' + i,
'android:enabled': 'true',
'android:icon': '@drawable/' + s.icon,
'android:shortcutShortLabel': '@string/shortcut_short_name_' + i,
'android:shortcutLongLabel': '@string/shortcut_name_' + i) {
'intent'(
'android:action': 'android.intent.action.MAIN',
'android:targetPackage': twaManifest.applicationId,
'android:targetClass': twaManifest.applicationId + '.LauncherActivity',
'android:data': s.url)
'categories'('android:name': 'android.intent.category.LAUNCHER')
}
}
}
shortcutsFile.text = xmlWriter.toString() + '\n'
}
preBuild.dependsOn(generateShorcutsFile)
repositories {
}
dependencies {
implementation fileTree(include: ['*.jar'], dir: 'libs')
implementation 'com.google.androidbrowserhelper:locationdelegation:1.1.1'
implementation 'com.google.androidbrowserhelper:androidbrowserhelper:2.5.0'
}

188
src/interface/android/app/src/main/AndroidManifest.xml Normal file
View File

@@ -0,0 +1,188 @@
<!--
Copyright 2019 Google Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- The "package" attribute is rewritten by the Gradle build with the value of applicationId.
It is still required here, as it is used to derive paths, for instance when referring
to an Activity by ".MyActivity" instead of the full name. If more Activities are added to the
application, the package attribute will need to reflect the correct path in order to use
the abbreviated format. -->
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="dev.khoj.app">
<uses-permission android:name="android.permission.POST_NOTIFICATIONS"/>
<application
android:name="Application"
android:allowBackup="true"
android:icon="@mipmap/ic_launcher"
android:label="@string/appName"
android:manageSpaceActivity="com.google.androidbrowserhelper.trusted.ManageDataLauncherActivity"
android:supportsRtl="true"
android:theme="@android:style/Theme.Translucent.NoTitleBar">
<meta-data
android:name="asset_statements"
android:resource="@string/assetStatements" />
<meta-data
android:name="web_manifest_url"
android:value="@string/webManifestUrl" />
<meta-data
android:name="twa_generator"
android:value="@string/generatorApp" />
<activity android:name="com.google.androidbrowserhelper.trusted.ManageDataLauncherActivity">
<meta-data
android:name="android.support.customtabs.trusted.MANAGE_SPACE_URL"
android:value="@string/launchUrl" />
</activity>
<activity android:name="LauncherActivity"
android:alwaysRetainTaskState="true"
android:label="@string/launcherName"
android:exported="true">
<meta-data android:name="android.support.customtabs.trusted.DEFAULT_URL"
android:value="@string/launchUrl" />
<meta-data
android:name="android.support.customtabs.trusted.STATUS_BAR_COLOR"
android:resource="@color/colorPrimary" />
<meta-data
android:name="android.support.customtabs.trusted.STATUS_BAR_COLOR_DARK"
android:resource="@color/colorPrimaryDark" />
<meta-data
android:name="android.support.customtabs.trusted.NAVIGATION_BAR_COLOR"
android:resource="@color/navigationColor" />
<meta-data
android:name="android.support.customtabs.trusted.NAVIGATION_BAR_COLOR_DARK"
android:resource="@color/navigationColorDark" />
<meta-data
android:name="androix.browser.trusted.NAVIGATION_BAR_DIVIDER_COLOR"
android:resource="@color/navigationDividerColor" />
<meta-data
android:name="androix.browser.trusted.NAVIGATION_BAR_DIVIDER_COLOR_DARK"
android:resource="@color/navigationDividerColorDark" />
<meta-data android:name="android.support.customtabs.trusted.SPLASH_IMAGE_DRAWABLE"
android:resource="@drawable/splash"/>
<meta-data android:name="android.support.customtabs.trusted.SPLASH_SCREEN_BACKGROUND_COLOR"
android:resource="@color/backgroundColor"/>
<meta-data android:name="android.support.customtabs.trusted.SPLASH_SCREEN_FADE_OUT_DURATION"
android:value="@integer/splashScreenFadeOutDuration"/>
<meta-data android:name="android.support.customtabs.trusted.FILE_PROVIDER_AUTHORITY"
android:value="@string/providerAuthority"/>
<meta-data android:name="android.app.shortcuts" android:resource="@xml/shortcuts" />
<meta-data android:name="android.support.customtabs.trusted.FALLBACK_STRATEGY"
android:value="@string/fallbackType" />
<meta-data android:name="android.support.customtabs.trusted.SCREEN_ORIENTATION"
android:value="@string/orientation"/>
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<intent-filter android:autoVerify="true">
<action android:name="android.intent.action.VIEW"/>
<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE"/>
<data android:scheme="https"
android:host="@string/hostName"/>
</intent-filter>
</activity>
<activity android:name="com.google.androidbrowserhelper.trusted.FocusActivity" />
<activity android:name="com.google.androidbrowserhelper.trusted.WebViewFallbackActivity"
android:configChanges="orientation|screenSize" />
<provider
android:name="androidx.core.content.FileProvider"
android:authorities="@string/providerAuthority"
android:grantUriPermissions="true"
android:exported="false">
<meta-data
android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/filepaths" />
</provider>
<service
android:name=".DelegationService"
android:enabled="@bool/enableNotification"
android:exported="@bool/enableNotification">
<meta-data
android:name="android.support.customtabs.trusted.SMALL_ICON"
android:resource="@drawable/ic_notification_icon" />
<intent-filter>
<action android:name="android.support.customtabs.trusted.TRUSTED_WEB_ACTIVITY_SERVICE"/>
<category android:name="android.intent.category.DEFAULT"/>
</intent-filter>
</service>
<activity android:name="com.google.androidbrowserhelper.trusted.NotificationPermissionRequestActivity" />
<activity android:name=
"com.google.androidbrowserhelper.locationdelegation.PermissionRequestActivity"/>
</application>
</manifest>

29
src/interface/android/app/src/main/java/dev/khoj/app/Application.java Normal file
View File

@@ -0,0 +1,29 @@
/*
* Copyright 2020 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package dev.khoj.app;
public class Application extends android.app.Application {
@Override
public void onCreate() {
super.onCreate();
}
}

17
src/interface/android/app/src/main/java/dev/khoj/app/DelegationService.java Normal file
View File

@@ -0,0 +1,17 @@
package dev.khoj.app;
import com.google.androidbrowserhelper.locationdelegation.LocationDelegationExtraCommandHandler;
public class DelegationService extends
com.google.androidbrowserhelper.trusted.DelegationService {
@Override
public void onCreate() {
super.onCreate();
registerExtraCommandHandler(new LocationDelegationExtraCommandHandler());
}
}

49
src/interface/android/app/src/main/java/dev/khoj/app/LauncherActivity.java Normal file
View File

@@ -0,0 +1,49 @@
/*
* Copyright 2020 Google Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package dev.khoj.app;
import android.content.pm.ActivityInfo;
import android.net.Uri;
import android.os.Bundle;
public class LauncherActivity
extends com.google.androidbrowserhelper.trusted.LauncherActivity {
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Setting an orientation crashes the app due to the transparent background on Android 8.0
// Oreo and below. We only set the orientation on Oreo and above. This only affects the
// splash screen and Chrome will still respect the orientation.
// See https://github.com/GoogleChromeLabs/bubblewrap/issues/496 for details.
setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED);
}
@Override
protected Uri getLaunchingUrl() {
// Get the original launch Url.
Uri uri = super.getLaunchingUrl();
return uri;
}
}

25
src/interface/android/app/src/main/res/drawable-anydpi/shortcut_legacy_background.xml Normal file
View File

@@ -0,0 +1,25 @@
<!--
Copyright 2020 Google Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<inset xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:aapt="http://schemas.android.com/aapt"
android:inset="2dp">
<aapt:attr name="android:drawable">
<shape android:shape="oval">
<solid android:color="@color/shortcut_background" />
<size android:width="44dp" android:height="44dp" />
</shape>
</aapt:attr>
</inset>

BIN
src/interface/android/app/src/main/res/drawable-hdpi/ic_notification_icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.1 KiB

BIN
src/interface/android/app/src/main/res/drawable-hdpi/splash.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 21 KiB

BIN
src/interface/android/app/src/main/res/drawable-mdpi/ic_notification_icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 678 B

BIN
src/interface/android/app/src/main/res/drawable-mdpi/splash.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

BIN
src/interface/android/app/src/main/res/drawable-xhdpi/ic_notification_icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.5 KiB

BIN
src/interface/android/app/src/main/res/drawable-xhdpi/splash.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 26 KiB

BIN
src/interface/android/app/src/main/res/drawable-xxhdpi/ic_notification_icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.4 KiB

BIN
src/interface/android/app/src/main/res/drawable-xxhdpi/splash.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 50 KiB

BIN
src/interface/android/app/src/main/res/drawable-xxxhdpi/ic_notification_icon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.4 KiB

BIN
src/interface/android/app/src/main/res/drawable-xxxhdpi/splash.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 60 KiB

BIN
src/interface/android/app/src/main/res/mipmap-hdpi/ic_launcher.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.0 KiB

BIN
src/interface/android/app/src/main/res/mipmap-mdpi/ic_launcher.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.7 KiB

BIN
src/interface/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.3 KiB

BIN
src/interface/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.0 KiB

BIN
src/interface/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 12 KiB

1
src/interface/android/app/src/main/res/raw/web_app_manifest.json Normal file
View File

@@ -0,0 +1 @@
{"name":"Khoj","short_name":"Khoj","display":"standalone","start_url":"/","description":"The open, personal AI for your digital brain. You can ask Khoj to draft a message, paint your imagination, find information on the internet and even answer questions from your documents.","theme_color":"#ffffff","background_color":"#ffffff","icons":[{"src":"/static/assets/icons/khoj_lantern_128x128.png","sizes":"128x128","type":"image/png"},{"src":"/static/assets/icons/khoj_lantern_256x256.png","sizes":"256x256","type":"image/png"}],"screenshots":[{"src":"/static/assets/samples/phone-remember-plan-sample.png","sizes":"419x900","type":"image/png","form_factor":"narrow","label":"Remember and Plan"},{"src":"/static/assets/samples/phone-browse-draw-sample.png","sizes":"419x900","type":"image/png","form_factor":"narrow","label":"Browse and Draw"},{"src":"/static/assets/samples/desktop-remember-plan-sample.png","sizes":"1260x742","type":"image/png","form_factor":"wide","label":"Remember and Plan"},{"src":"/static/assets/samples/desktop-browse-draw-sample.png","sizes":"1260x742","type":"image/png","form_factor":"wide","label":"Browse and Draw"}]}

Some files were not shown because too many files have changed in this diff Show More