mirror of
https://github.com/tradecatlabs/vibe-coding-cn.git
synced 2026-08-18 21:38:05 +00:00
chore: migrate repository to standard knowledge base layout
This commit is contained in:
tukuaiai
@@ -0,0 +1,38 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
venv/
|
||||
env/
|
||||
ENV/
|
||||
|
||||
# 凭据和密钥
|
||||
credentials.json
|
||||
token.json
|
||||
*.key
|
||||
.env
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# 临时文件
|
||||
*.tmp
|
||||
*.bak
|
||||
*.log
|
||||
temp/
|
||||
|
||||
# Excel临时文件
|
||||
~$*.xlsx
|
||||
~$*.xls
|
||||
|
||||
# 协作记录文件
|
||||
ticket.md
|
||||
@@ -0,0 +1,278 @@
|
||||
<!--
|
||||
-------------------------------------------------------------------------------
|
||||
项目头部区域 (HEADER)
|
||||
这是用户第一眼看到的地方。一个精美的横幅或 Logo 能立刻提升专业感。
|
||||
-------------------------------------------------------------------------------
|
||||
-->
|
||||
<p align="center">
|
||||
<!-- 建议尺寸: 1280x640px。可以使用 Canva, Figma 或 https://banners.beyondco.de/ 等工具制作 -->
|
||||
<img src="https://github.com/tukuaiai.png" alt="项目横幅">
|
||||
</p>
|
||||
|
||||
<div align="center">
|
||||
|
||||
# Prompt Library
|
||||
|
||||
**一个全面的高质量AI提示词库，支持Excel和Markdown格式之间的双向转换。**
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
徽章区域 (BADGES)
|
||||
从 https://shields.io/ 生成。选择与你的品牌色一致的颜色。
|
||||
建议包含：构建状态, 版本, 许可证, 语言, 代码大小, 下载量, 社区链接等。
|
||||
-->
|
||||
<p>
|
||||
<a href="https://github.com/tukuaiai/prompt-library/actions/workflows/sync.yml"><img src="https://img.shields.io/github/actions/workflow/status/tukuaiai/prompt-library/sync.yml?style=for-the-badge" alt="构建状态"></a>
|
||||
<a href="https://github.com/tukuaiai/prompt-library/releases"><img src="https://img.shields.io/github/v/release/tukuaiai/prompt-library?style=for-the-badge" alt="最新版本"></a>
|
||||
<a href="https://github.com/tukuaiai/prompt-library/blob/main/LICENSE"><img src="https://img.shields.io/github/license/tukuaiai/prompt-library?style=for-the-badge" alt="许可证"></a>
|
||||
<a href="https://github.com/tukuaiai/prompt-library"><img src="https://img.shields.io/github/languages/top/tukuaiai/prompt-library?style=for-the-badge" alt="主要语言"></a>
|
||||
<a href="https://github.com/tukuaiai/prompt-library"><img src="https://img.shields.io/github/languages/code-size/tukuaiai/prompt-library?style=for-the-badge" alt="代码大小"></a>
|
||||
</p>
|
||||
|
||||
[✨ 功能特性](#-功能特性) •
|
||||
[🚀 快速开始](#-快速开始) •
|
||||
[⚙️ API参考](#️-api参考) •
|
||||
[🤝 参与贡献](#-参与贡献) •
|
||||
[🗺️ 路线图](#️-路线图)
|
||||
|
||||
</div>
|
||||
|
||||
<!--
|
||||
-------------------------------------------------------------------------------
|
||||
概览与核心视觉区域 (OVERVIEW & VISUALS)
|
||||
展示项目最直观的部分。
|
||||
-------------------------------------------------------------------------------
|
||||
-->
|
||||
|
||||
---
|
||||
|
||||
## 🖼️ 概览与演示
|
||||
|
||||
本词库是一个结构化的、高质量的AI提示词（Prompts）集合，旨在为开发人员、研究人员和内容创作者提供强大而灵活的工具。与许多静态的提示词列表不同，本项目提供了一个双向转换的工作流程，允许用户在易于协作的Excel格式和便于版本控制的Markdown格式之间无缝切换。
|
||||
|
||||
> **核心理念**: 让高质量的提示词像代码一样被管理、分享和迭代。
|
||||
|
||||
<p align="center">
|
||||
<!-- 建议使用 GIF，工具: ScreenToGif, GIPHY CAPTURE -->
|
||||
<img src="https://user-images.githubusercontent.com/12523395/269150161-1a83689c-2f3a-4a0e-8d18-114cec03be8d.gif" alt="项目演示动画" width="80%">
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
-------------------------------------------------------------------------------
|
||||
主要内容区域 (MAIN CONTENT)
|
||||
详细介绍项目的方方面面。
|
||||
-------------------------------------------------------------------------------
|
||||
-->
|
||||
|
||||
## ✨ 功能特性
|
||||
|
||||
- 📊 **双向转换**: 支持Excel ↔️ Markdown格式互转，兼顾易用性与版本控制。
|
||||
- 🗂️ **结构化管理**: 包含多个分类，覆盖从软件工程到哲学思辨的广泛领域。
|
||||
- 🤖 **多平台兼容**: 提示词设计兼容Claude、GPT、Gemini等主流AI模型。
|
||||
- 🛠️ **自动化工具**: 提供命令行工具，支持批量转换和管理。
|
||||
- 🎨 **易于扩展**: 可以方便地添加新的提示词、分类和自定义属性。
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ 架构与工作流程
|
||||
|
||||
本项目的工作流程围绕“以结构化数据为中心”的思想构建。
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph "Excel-First 工作流"
|
||||
A[1. 在 Excel 中编辑提示词] --> B{运行转换脚本};
|
||||
end
|
||||
|
||||
subgraph "Git-Native 工作流"
|
||||
D[3. 在 Markdown 文件中编辑] --> E{运行转换脚本};
|
||||
end
|
||||
|
||||
subgraph "中央产物"
|
||||
B --> C[2. 生成结构化的 Markdown 文件];
|
||||
E --> F[4. 更新或生成 Excel 文件]
|
||||
end
|
||||
|
||||
style A fill:#D5E8D4
|
||||
style D fill:#DAE8FC
|
||||
style C fill:#F8CECC
|
||||
style F fill:#F8CECC
|
||||
```
|
||||
这个流程确保了无论是喜欢电子表格的非技术人员，还是习惯于Git和代码编辑器的开发人员，都可以高效地协作。
|
||||
|
||||
---
|
||||
|
||||
## 🚀 快速开始
|
||||
|
||||
### 1. 环境依赖
|
||||
|
||||
- [Python](https://www.python.org/) >= 3.8
|
||||
|
||||
### 2. 安装
|
||||
|
||||
<details>
|
||||
<summary><b>从源码构建</b></summary>
|
||||
|
||||
```bash
|
||||
git clone https://github.com/tukuaiai/prompt-library.git
|
||||
cd prompt-library
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
</details>
|
||||
|
||||
### 3. 使用
|
||||
|
||||
<details>
|
||||
<summary><b>Excel → Markdown 转换</b></summary>
|
||||
|
||||
```bash
|
||||
# 运行交互式转换
|
||||
python3 main.py
|
||||
```
|
||||
程序将扫描 `prompt_excel` 目录下的 `.xlsx` 文件，并让你选择一个进行转换。结果将输出到 `prompt_docs` 目录下一个带时间戳的文件夹中。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Markdown → Excel 转换</b></summary>
|
||||
|
||||
```bash
|
||||
# 运行交互式转换
|
||||
python3 main.py
|
||||
```
|
||||
程序将扫描 `prompt_docs` 目录，让你选择一个文档集，然后将其转换回 Excel 文件，并输出到 `prompt_excel` 目录下一个带时间戳的文件夹中。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>非交互式转换</b></summary>
|
||||
|
||||
```bash
|
||||
# 指定要转换的 Excel 文件
|
||||
python3 main.py --select "prompt_excel/your_file.xlsx"
|
||||
|
||||
# 指定要转换的 Markdown 目录
|
||||
python3 main.py --select "prompt_docs/your_docs_folder"
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Gemini 无头：Markdown → JSONL 批处理</b></summary>
|
||||
|
||||
```bash
|
||||
# 将目录中的 .md 提示词批量转为 JSONL（默认输入 2/，输出 2/prompts.jsonl）
|
||||
python3 scripts/gemini_jsonl_batch.py --input 2 --output 2/prompts.jsonl --model gemini-2.5-flash
|
||||
```
|
||||
|
||||
- 脚本内置固定系统提示词，调用 `gemini --allowed-tools '' --output-format text` 确保纯文本 JSONL。
|
||||
- 需要本地已登录的 Gemini CLI；如需代理，请自行设置 `http_proxy/https_proxy` 环境变量。
|
||||
- 可用 `-v` 查看逐文件处理日志，`--gemini-cmd` 自定义 CLI 可执行路径。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>内部 JSONL Excel → JSONL 目录</b></summary>
|
||||
|
||||
```bash
|
||||
# 将内部 JSONL 格式的 xlsx 按工作表拆分为多个 jsonl 文件
|
||||
python3 main.py --select "prompt_excel/prompt_jsonl.xlsx" --mode jsonl_excel2jsonl
|
||||
```
|
||||
|
||||
- 输出目录格式为 `prompt_jsonl/<excel_name>_<timestamp>/`
|
||||
- 每个工作表输出一个独立的 `.jsonl` 文件
|
||||
- 文件名格式为 `<序号>_<sheet_name>.jsonl`
|
||||
- 若工作表中没有标准 JSON 单元格，会对纯文本单元格做 JSONL 兜底转换
|
||||
- 自动忽略名为 `说明` 的工作表
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
<details>
|
||||
<summary>❓ 常见问题 (FAQ) (可选)</summary>
|
||||
|
||||
- **Q: 为什么转换会失败？**
|
||||
- **A:** 请确保您的Excel文件格式与提供的示例一致，特别是工作表（Sheet）的名称和列的标题。
|
||||
|
||||
- **Q: 我可以添加自己的转换逻辑吗？**
|
||||
- **A:** 当然可以。核心逻辑位于 `scripts/` 目录下，您可以自由修改或扩展它们。
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
-------------------------------------------------------------------------------
|
||||
社区与治理区域 (COMMUNITY & GOVERNANCE)
|
||||
展示项目的健康度和发展方向。
|
||||
-------------------------------------------------------------------------------
|
||||
-->
|
||||
|
||||
## 🗺️ 路线图
|
||||
|
||||
```mermaid
|
||||
gantt
|
||||
title 项目发展路线图
|
||||
dateFormat YYYY-MM
|
||||
section 核心功能
|
||||
双向转换脚本 :done, 2024-10, 30d
|
||||
交互式CLI :done, 2024-11, 20d
|
||||
section 未来计划
|
||||
Web界面 :2025-01, 30d
|
||||
更丰富的导出格式 :2025-02, 20d
|
||||
云同步功能 : 2025-03, 30d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🤝 参与贡献
|
||||
|
||||
我们热烈欢迎各种形式的贡献！如果您对本项目有任何想法或建议，请随时开启一个 [Issue](https://github.com/tukuaiai/prompt-library/issues) 或提交一个 [Pull Request](https://github.com/tukuaiai/prompt-library/pulls)。
|
||||
|
||||
在您开始之前，请花点时间阅读我们的 [**贡献指南 (CONTRIBUTING.md)**](https://github.com/tukuaiai/prompt-library/blob/main/CONTRIBUTING.md) 和 [**行为准则 (CODE_OF_CONDUCT.md)**](https://github.com/tukuaiai/prompt-library/blob/main/CODE_OF_CONDUCT.md)。
|
||||
|
||||
### ✨ 贡献者们
|
||||
|
||||
感谢所有为本项目做出贡献的开发者！
|
||||
|
||||
<a href="https://github.com/tukuaiai/prompt-library/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=tukuaiai/prompt-library" />
|
||||
</a>
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ 安全策略
|
||||
|
||||
我们非常重视项目的安全性。如果您发现了任何安全漏洞，请不要公开讨论，而是通过电子邮件 `tukuaiai@example.com` 与我们联系。
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
-------------------------------------------------------------------------------
|
||||
页脚区域 (FOOTER)
|
||||
最后的行动号召和感谢。
|
||||
-------------------------------------------------------------------------------
|
||||
-->
|
||||
|
||||
## 📜 许可证
|
||||
|
||||
本项目采用 [MIT](https://github.com/tukuaiai/prompt-library/blob/main/LICENSE) 许可证。
|
||||
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
|
||||
**如果这个项目对您有帮助，请不要吝啬您的 Star ⭐！**
|
||||
|
||||
<!-- Star History: https://star-history.com/ -->
|
||||
<a href="https://star-history.com/#tukuaiai/prompt-library&Date">
|
||||
<img src="https://api.star-history.com/svg?repos=tukuaiai/prompt-library&type=Date" alt="Star History Chart" width="80%">
|
||||
</a>
|
||||
|
||||
<br>
|
||||
|
||||
**Made with ❤️ by tukuaiai**
|
||||
|
||||
[⬆ 回到顶部](#prompt-library)
|
||||
|
||||
</div>
|
||||
@@ -0,0 +1,56 @@
|
||||
# JSONL 提示词转换器 - 系统提示词
|
||||
|
||||
你是一个专业的提示词格式转换器。将用户提供的提示词内容转换为标准 JSONL 格式。
|
||||
|
||||
## 输出格式
|
||||
|
||||
```json
|
||||
{"title": "<标题>", "content": "<完整内容>"}
|
||||
```
|
||||
|
||||
### 字段说明
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `title` | string | 提示词标题，取内容的第一行或前 50 字符 |
|
||||
| `content` | string | 完整的提示词内容 |
|
||||
|
||||
## 转换规则
|
||||
|
||||
1. **标题提取**：
|
||||
- 若内容以 `#` 开头，取第一个标题作为 title
|
||||
- 否则取前 50 字符（去除换行）
|
||||
2. **内容转义**：
|
||||
- 换行符转为 `\n`
|
||||
- 双引号转为 `\"`
|
||||
- 反斜杠转为 `\\`
|
||||
|
||||
## 输出要求
|
||||
|
||||
- 每行一个完整的 JSON 对象
|
||||
- 不要添加任何解释、注释或额外文字
|
||||
- 不要用 ```json 代码块包裹
|
||||
- 直接输出纯 JSONL 内容
|
||||
|
||||
## 示例
|
||||
|
||||
### 输入
|
||||
```
|
||||
# Role：智能文档助手
|
||||
|
||||
## Background
|
||||
用户需要一个能够处理文档的 AI 助手。
|
||||
|
||||
## Skills
|
||||
- 文档解析
|
||||
- 格式转换
|
||||
```
|
||||
|
||||
### 输出
|
||||
```
|
||||
{"title": "# Role：智能文档助手", "content": "# Role：智能文档助手\n\n## Background\n用户需要一个能够处理文档的 AI 助手。\n\n## Skills\n- 文档解析\n- 格式转换"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
现在，请将用户提供的内容转换为标准 JSONL 格式。
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,38 @@
|
||||
# 💰 项目支持（从Excel提取）
|
||||
|
||||
|
||||
## 支持说明
|
||||
**礼貌要饭地址** - 如果这个项目对您有帮助，欢迎通过以下方式支持
|
||||
|
||||
|
||||
## 加密货币钱包地址
|
||||
|
||||
### 主流网络支持
|
||||
|
||||
|
||||
| 网络名称 | 钱包地址 | Excel行号 |
|
||||
|----------|----------|-----------|
|
||||
|
||||
| **TRON** | `TQtBXCSTwLFHjBqTS4rNUp7ufiGx51BRey` | 第12行 |
|
||||
|
||||
| **SOL** | `HjYhozVf9AQmfv7yv79xSNs6uaEU5oUk2USasYQfUYau` | 第13行 |
|
||||
|
||||
| **ETH** | `0xa396923a71ee7D9480b346a17dDeEb2c0C287BBC` | 第14行 |
|
||||
|
||||
| **BSC** | `0xa396923a71ee7D9480b346a17dDeEb2c0C287BBC` | 第15行 |
|
||||
|
||||
| **BTC** | `bc1plslluj3zq3snpnnczplu7ywf37h89dyudqua04pz4txwh8z5z5vsre7nlm` | 第16行 |
|
||||
|
||||
| **SUI** | `0xb720c98a48c77f2d49d375932b2867e793029e6337f1562522640e4f84203d2e` | 第17行 |
|
||||
|
||||
|
||||
⚠️ **重要提醒**: 广告位（注意识别风险）
|
||||
|
||||
|
||||
### 使用建议
|
||||
1. 请确认钱包地址的准确性
|
||||
2. 建议小额测试后再进行大额转账
|
||||
3. 不同网络的转账费用不同，请选择合适的网络
|
||||
|
||||
---
|
||||
*钱包地址来源: prompt (3).xlsx*
|
||||
@@ -0,0 +1,35 @@
|
||||
# 🛠️ 工具与资源（从Excel提取）
|
||||
|
||||
|
||||
## AI优化工具
|
||||
|
||||
|
||||
### OpenAI 提示词优化平台
|
||||
- **URL**: https://platform.openai.com/chat/edit?models=gpt-5&optimize=true
|
||||
- **描述**: openai提示词优化网站
|
||||
- **数据来源**: Excel表格第7行
|
||||
|
||||
|
||||
### 工具
|
||||
- **URL**: https://aistudio.google.com/
|
||||
- **描述**: 打开 Gemini 3.1 Pro
|
||||
- **数据来源**: Excel表格第4行
|
||||
|
||||
|
||||
## 社交媒体
|
||||
|
||||
|
||||
### Twitter/X 账号
|
||||
- **URL**: https://x.com/123olp
|
||||
- **描述**: 点击关注我的推特，获取最新动态，首页接广告位
|
||||
- **数据来源**: Excel表格第9行
|
||||
|
||||
|
||||
## 使用建议
|
||||
|
||||
1. **OpenAI优化器**: 可以用来测试和改进本库中的提示词
|
||||
2. **社交媒体**: 关注获取项目更新和使用技巧
|
||||
3. **集成方式**: 可以将这些工具集成到自动化工作流中
|
||||
|
||||
---
|
||||
*数据来源: prompt (3).xlsx*
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,127 @@
|
||||
# 提示词库结构与 Excel 互转规范
|
||||
|
||||
> 本规范用于约束“提示词库”在文件系统与 Excel 之间的一致结构、命名与转换规则，确保后期自动化同步、增量更新与团队协作的稳定性。
|
||||
|
||||
## 1. 目录结构（约定）
|
||||
|
||||
```
|
||||
prompt-library/
|
||||
├── prompts/ # 核心输出目录
|
||||
│ ├── <工作表名称-已净化>/ # 每个 Excel 子表对应一个文件夹
|
||||
│ │ ├── (行,列)_标题.md # 单元格 → 单文件（仅包含提示词文本）
|
||||
│ │ └── index.md # 该表分类索引（自动生成）
|
||||
│ └── index.json # 全局 JSON 索引（自动生成）
|
||||
├── docs/
|
||||
│ ├── tools.md # 工具/链接（从表自动提取）
|
||||
│ ├── support.md # 支持/钱包地址（从表自动提取）
|
||||
│ ├── excel-data.md # Excel 原始数据快照（自动生成）
|
||||
│ └── STRUCTURE_AND_CONVERSION_SPEC.md # 本规范
|
||||
└── scripts/
|
||||
├── convert_local.py # Excel ↔ 文件 的本地转换器（实现 Excel→文件）
|
||||
├── config.yaml # 可选配置（数据源/策略）
|
||||
└── requirements.txt # 依赖
|
||||
```
|
||||
|
||||
- 只允许在 `prompts/<工作表名称-已净化>/` 下新增/修改提示词文件，其它自动文件（如 `index.md`、`index.json`）由转换器生成，禁止手改。
|
||||
|
||||
## 2. 命名与净化（Sanitization）
|
||||
|
||||
- 工作表文件夹名：对原始 Excel 工作表名称执行净化：
|
||||
- 移除非法字符：`\\ / : * ? " < > | \r \n`
|
||||
- 将空格替换为下划线 `_`
|
||||
- 最长 60 字符，超长截断
|
||||
- 示例：`"学习 提示词/1?"` → `"学习_提示词1"`
|
||||
- 提示词文件名：`(行,列)_标题.md`
|
||||
- 行、列为 1 基索引（Excel 中第 1 行/第 1 列即 1）
|
||||
- 标题来源：该行第一个非空单元格的首行文本的前若干词（用于人读），同样执行净化
|
||||
- 标题仅用于文件名可读性，回写 Excel 时忽略
|
||||
|
||||
## 3. 文件内容规范（强约束）
|
||||
|
||||
- 每个提示词文件（.md）内容必须是“对应单元格的原始文本”，末尾追加一个换行；不允许包含：
|
||||
- Markdown 标题/分隔线/元信息
|
||||
- 代码围栏（```）
|
||||
- 版本历史/导航/时间戳
|
||||
- 编码：UTF-8，换行：LF
|
||||
|
||||
## 4. Excel → 文件 的生成规则
|
||||
|
||||
- 每个工作表 → `prompts/<工作表名称-已净化>/`
|
||||
- 行类型判定（逐行）：
|
||||
- 提示词行：该行至少一列非空，且不属于工具/社交/钱包/警告/占位
|
||||
- 工具/链接：同行包含 `http` 且非 X/Twitter 链接 → 归档 `docs/tools.md`
|
||||
- 社交：链接中包含 `x.com`/`twitter.com` → 归档 `docs/tools.md`
|
||||
- 钱包区块：先出现“表头”（含“网络/网络名称”且含“礼貌要饭地址/钱包/地址”），其后若干行按 `(网络, 地址)` 提取 → 归档 `docs/support.md`
|
||||
- 警告：包含“广告位” → 记录为警告
|
||||
- 占位：`...`/`….`/`....` → 忽略
|
||||
- 对每个提示词行：
|
||||
- 对该行每个非空列 `c` 生成：`(r,c)_标题.md`，文件正文 = 单元格纯文本
|
||||
- 自动索引：
|
||||
- `prompts/<表>/index.md`：统计、列表与版本矩阵（供浏览用，自动生成）
|
||||
- `prompts/index.json`：全局结构化索引，含各表、各行版本与外部资源（自动生成）
|
||||
- `docs/excel-data.md`：各表数据快照（截取前 3 列），便于审计
|
||||
|
||||
## 5. 文件 → Excel 的回写规则（规范定义，供实现）
|
||||
|
||||
> 说明：当前脚本已实现 Excel→文件；本节定义未来“文件→Excel”时的规范，以便实现反向同步器保持一致性。
|
||||
|
||||
- 工作表名：优先使用 `prompts/index.json` 中的 `categories[].name` 作为原始工作表名；若缺失，采用文件夹名将 `_` 还原为空格（尽力而为）。
|
||||
- 单元格位置：
|
||||
- 解析文件名 `(r,c)_标题.md` 得到 1 基 `r,c`，写入 Excel 的第 `r` 行、第 `c` 列
|
||||
- 标题部分忽略，仅用于文件名可读
|
||||
- 写入值：文件全文（去掉末尾多余空行）即单元格值
|
||||
- 缺表/缺行列：自动创建工作表、扩展行列
|
||||
- 冲突：同一 `(r,c)` 出现多文件时应失败并报告（推荐），或采用“最后写入覆盖”（需在实现中明确开关）
|
||||
- 非提示词文档：`tools.md`、`support.md` 不默认回写；如需回写，应定义目标工作表名称与行布局（建议启用配置项）
|
||||
|
||||
## 6. 变更与校验
|
||||
|
||||
- 合法性检查：
|
||||
- 文件名需符合正则：`^\(\d+,\d+\)_.+\.md$`
|
||||
- 内容不得包含本规范禁止的附加信息（元信息、代码围栏等）
|
||||
- 冲突检测：同一 `(r,c)` 不允许多次定义
|
||||
- 兼容性：文件夹名与表名的映射以 `index.json` 为准；直接根据文件夹名还原可能出偏差
|
||||
|
||||
## 7. 自动化工具与用法
|
||||
|
||||
- 依赖安装：
|
||||
```bash
|
||||
python3 -m pip install -r prompt-library/scripts/requirements.txt
|
||||
```
|
||||
- 运行（Excel→文件）：
|
||||
```bash
|
||||
python3 "prompt-library/scripts/convert_local.py" \
|
||||
--excel "prompt (2).xlsx"
|
||||
```
|
||||
- 行为摘要：
|
||||
- 为每个工作表生成一个同名（净化后）目录
|
||||
- 为每个非空单元格生成一个 `.md` 文件（仅含单元格文本）
|
||||
- 重新生成所有索引与快照文档
|
||||
|
||||
## 8. 新增提示词（两种路径）
|
||||
|
||||
- 在 Excel 中新增：
|
||||
- 在目标工作表的目标行、列填写文本
|
||||
- 运行转换脚本，生成对应 `(行,列)_标题.md`
|
||||
- 在文件系统中新增：
|
||||
- 到 `prompts/<工作表名称-已净化>/` 新建符合命名规范的文件
|
||||
- 文件正文仅填写提示词文本
|
||||
- 将在“文件→Excel”实现后由反向同步器写回相应 `(行,列)`
|
||||
|
||||
## 9. 设计取舍（Why）
|
||||
|
||||
- 只在提示词文件中保留“纯内容”，避免元信息造成噪声与二义性，便于复制/拼接/训练
|
||||
- 元信息、导航、统计统一由自动化生成到索引与文档中，降低手工维护成本
|
||||
- 以 `(行,列)` 作为跨介质的“主键”，确保 Excel 与文件的可逆映射
|
||||
|
||||
## 10. 版本与扩展
|
||||
|
||||
- 建议在 `scripts/config.yaml` 中引入可选项：
|
||||
- 反向同步开关与目标工作表名策略
|
||||
- 冲突策略（报错/覆盖）
|
||||
- 过滤器（忽略某些行/列/关键词）
|
||||
- 未来可扩展：GitHub Actions/CI 校验规范合规性；支持更多字段映射（如 tag、语言）。
|
||||
|
||||
---
|
||||
|
||||
本规范适用于本仓库的所有提示词数据。如需调整，请在 PR 中同步更新 `docs/STRUCTURE_AND_CONVERSION_SPEC.md` 并说明向后兼容策略。
|
||||
@@ -0,0 +1,624 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
r"""
|
||||
main.py
|
||||
|
||||
Unified controller for prompt-library conversions.
|
||||
|
||||
支持的转换模式
|
||||
==============
|
||||
1. Excel → Docs : 将 Excel 工作簿转换为 Markdown 文档目录
|
||||
2. Docs → Excel : 将 Markdown 文档目录还原为 Excel 工作簿
|
||||
3. Docs → JSONL : 将 Markdown 文档转换为 JSONL 格式（保留完整元信息）
|
||||
4. JSONL → Excel : 将 JSONL 转换为 Excel（单元格存储 JSON 对象）
|
||||
5. Excel(JSONL) → JSONL : 将内部 JSONL 格式的 Excel 转换为 JSONL 目录（自动忽略"说明"工作表）
|
||||
|
||||
数据格式规范
|
||||
============
|
||||
Excel 结构:
|
||||
- 每个工作表(sheet) = 一个分类(category)
|
||||
- 行(row) = 不同提示词
|
||||
- 列(col) = 版本迭代
|
||||
|
||||
Excel(JSONL) 结构（内部 JSONL 格式）:
|
||||
- 每个工作表(sheet) = 一个分类(category)，"说明"工作表会被忽略
|
||||
- 每个单元格存储 JSON 对象: {"title": "...", "content": "..."}
|
||||
|
||||
Docs 结构:
|
||||
- prompts/(N)_分类名/ # N = category_id
|
||||
- prompts/(N)_分类名/(r,c)_标题.md # r=row, c=col
|
||||
|
||||
JSONL 格式 (每行一个 JSON 对象):
|
||||
{
|
||||
"category_id": 2, # 分类编号
|
||||
"category": "元提示词", # 分类名称
|
||||
"row": 1, # 原 Excel 行号
|
||||
"col": 1, # 原 Excel 列号（版本号）
|
||||
"title": "...", # 标题（截断80字符）
|
||||
"content": "..." # 完整内容
|
||||
}
|
||||
|
||||
JSONL → Excel 单元格格式:
|
||||
{"title": "...", "content": "..."} # 只保留 title 和 content
|
||||
|
||||
目录约定
|
||||
========
|
||||
- Excel 源文件: ./prompt_excel/
|
||||
- Docs 源目录: ./prompt_docs/
|
||||
- JSONL 文件: ./prompt_jsonl/
|
||||
- 输出:
|
||||
- Excel→Docs: ./prompt_docs/prompt_docs_YYYY_MMDD_HHMMSS/
|
||||
- Docs→Excel: ./prompt_excel/prompt_excel_YYYY_MMDD_HHMMSS/rebuilt.xlsx
|
||||
- Docs→JSONL: ./prompt_jsonl/{docs_name}.jsonl
|
||||
- JSONL→Excel: ./prompt_excel/{jsonl_name}.xlsx
|
||||
- Excel(JSONL)→JSONL: ./prompt_jsonl/{excel_name}_{timestamp}/<sheet>.jsonl
|
||||
|
||||
使用示例
|
||||
========
|
||||
# 交互式选择
|
||||
python3 main.py
|
||||
|
||||
# Excel → Docs
|
||||
python3 main.py --select "prompt_excel/prompt.xlsx"
|
||||
|
||||
# Docs → Excel
|
||||
python3 main.py --select "prompt_docs/prompt_docs_2025_1222"
|
||||
|
||||
# Docs → JSONL
|
||||
python3 main.py --select "prompt_docs/prompt_docs_2025_1222" --mode docs2jsonl
|
||||
|
||||
# JSONL → Excel
|
||||
python3 main.py --select "prompt_jsonl/prompt_docs.jsonl"
|
||||
|
||||
# Excel(JSONL) → JSONL（自动检测或显式指定）
|
||||
python3 main.py --select "prompt_excel/prompt_jsonl.xlsx"
|
||||
python3 main.py --select "prompt_excel/prompt_jsonl.xlsx" --mode jsonl_excel2jsonl
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Sequence, Tuple
|
||||
|
||||
# Optional Rich UI imports (fallback to plain if unavailable)
|
||||
try:
|
||||
from rich.console import Console
|
||||
from rich.layout import Layout
|
||||
from rich.panel import Panel
|
||||
from rich.table import Table
|
||||
from rich.text import Text
|
||||
from rich import box
|
||||
from rich.prompt import IntPrompt
|
||||
_RICH_AVAILABLE = True
|
||||
except Exception: # pragma: no cover
|
||||
_RICH_AVAILABLE = False
|
||||
|
||||
# Optional InquirerPy for arrow-key selection
|
||||
try:
|
||||
from InquirerPy import inquirer as _inq
|
||||
_INQUIRER_AVAILABLE = True
|
||||
except Exception: # pragma: no cover
|
||||
_INQUIRER_AVAILABLE = False
|
||||
|
||||
|
||||
@dataclass
|
||||
class Candidate:
|
||||
index: int
|
||||
kind: str # "excel" | "docs" | "docs2jsonl" | "jsonl"
|
||||
path: Path
|
||||
label: str
|
||||
|
||||
|
||||
def get_repo_root() -> Path:
|
||||
return Path(__file__).resolve().parent
|
||||
|
||||
|
||||
def list_excel_files(excel_dir: Path) -> List[Path]:
|
||||
if not excel_dir.exists():
|
||||
return []
|
||||
return sorted([p for p in excel_dir.iterdir() if p.is_file() and p.suffix.lower() == ".xlsx"], key=lambda p: p.stat().st_mtime)
|
||||
|
||||
|
||||
def has_prompt_files(directory: Path) -> bool:
|
||||
if not directory.exists():
|
||||
return False
|
||||
# Detect files like "(r,c)_*.md" anywhere under the directory
|
||||
for file_path in directory.rglob("*.md"):
|
||||
name = file_path.name
|
||||
if name.startswith("(") and ")_" in name:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def list_doc_sets(docs_dir: Path) -> List[Path]:
|
||||
results: List[Path] = []
|
||||
if not docs_dir.exists():
|
||||
return results
|
||||
# If the docs_dir itself looks like a set, include it
|
||||
if has_prompt_files(docs_dir):
|
||||
results.append(docs_dir)
|
||||
# Also include any immediate children that look like a docs set
|
||||
for child in sorted(docs_dir.iterdir()):
|
||||
if child.is_dir() and has_prompt_files(child):
|
||||
results.append(child)
|
||||
return results
|
||||
|
||||
|
||||
def run_start_convert(start_convert: Path, mode: str, project_root: Path, select_path: Optional[Path] = None, excel_dir: Optional[Path] = None, docs_dir: Optional[Path] = None) -> int:
|
||||
"""Delegate to scripts/start_convert.py with appropriate flags."""
|
||||
python_exe = sys.executable
|
||||
cmd: List[str] = [python_exe, str(start_convert), "--mode", mode]
|
||||
if select_path is not None:
|
||||
# Always pass as repo-root-relative or absolute string
|
||||
cmd.extend(["--select", str(select_path)])
|
||||
if excel_dir is not None:
|
||||
cmd.extend(["--excel-dir", str(excel_dir)])
|
||||
if docs_dir is not None:
|
||||
cmd.extend(["--docs-dir", str(docs_dir)])
|
||||
|
||||
# Execute in repo root to ensure relative defaults resolve correctly
|
||||
proc = subprocess.run(cmd, cwd=str(project_root))
|
||||
return proc.returncode
|
||||
|
||||
|
||||
def run_docs_to_jsonl(docs_path: Path, project_root: Path) -> int:
|
||||
"""Convert docs folder to JSONL format."""
|
||||
import json
|
||||
import re
|
||||
|
||||
prompts_dir = docs_path / "prompts"
|
||||
if not prompts_dir.exists():
|
||||
print(f"❌ 找不到 prompts 目录: {prompts_dir}")
|
||||
return 1
|
||||
|
||||
output_dir = project_root / "prompt_jsonl"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
output_file = output_dir / f"{docs_path.name}.jsonl"
|
||||
|
||||
records = []
|
||||
for category_dir in sorted(prompts_dir.iterdir()):
|
||||
if not category_dir.is_dir():
|
||||
continue
|
||||
|
||||
m = re.match(r'\((\d+)\)_(.+)', category_dir.name)
|
||||
cat_id, cat_name = (m.groups() if m else (0, category_dir.name))
|
||||
|
||||
for md_file in sorted(category_dir.glob("*.md")):
|
||||
if md_file.name == "index.md":
|
||||
continue
|
||||
|
||||
fm = re.match(r'\((\d+),(\d+)\)_(.+)\.md', md_file.name)
|
||||
if not fm:
|
||||
continue
|
||||
|
||||
row, col, title = fm.groups()
|
||||
content = md_file.read_text(encoding='utf-8')
|
||||
|
||||
records.append({
|
||||
"category_id": int(cat_id),
|
||||
"category": cat_name,
|
||||
"row": int(row),
|
||||
"col": int(col),
|
||||
"title": title[:80],
|
||||
"content": content
|
||||
})
|
||||
|
||||
with open(output_file, 'w', encoding='utf-8') as f:
|
||||
for r in records:
|
||||
f.write(json.dumps(r, ensure_ascii=False) + '\n')
|
||||
|
||||
print(f"✅ Docs→JSONL OK: {docs_path.name} → {output_file.relative_to(project_root)}")
|
||||
return 0
|
||||
|
||||
|
||||
def list_jsonl_files(jsonl_dir: Path) -> List[Path]:
|
||||
if not jsonl_dir.exists():
|
||||
return []
|
||||
return sorted([p for p in jsonl_dir.iterdir() if p.is_file() and p.suffix.lower() == ".jsonl"], key=lambda p: p.stat().st_mtime)
|
||||
|
||||
|
||||
def is_jsonl_excel(excel_path: Path) -> bool:
|
||||
"""检测 Excel 是否为内部 JSONL 格式（单元格存储 JSON 对象）"""
|
||||
import json
|
||||
try:
|
||||
import pandas as pd
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
try:
|
||||
xlsx = pd.ExcelFile(excel_path)
|
||||
for sheet in xlsx.sheet_names[:2]: # 检查前两个工作表
|
||||
if sheet == '说明':
|
||||
continue
|
||||
df = pd.read_excel(xlsx, sheet_name=sheet, header=None, nrows=1)
|
||||
if df.empty:
|
||||
continue
|
||||
first_val = str(df.iloc[0, 0]).strip() if not pd.isna(df.iloc[0, 0]) else ""
|
||||
# 检查列名或第一个单元格是否为 JSON
|
||||
first_col = str(df.columns[0]).strip() if len(df.columns) > 0 else ""
|
||||
for val in [first_col, first_val]:
|
||||
if val.startswith('{') and val.endswith('}'):
|
||||
try:
|
||||
obj = json.loads(val)
|
||||
if 'title' in obj and 'content' in obj:
|
||||
return True
|
||||
except:
|
||||
pass
|
||||
return False
|
||||
except:
|
||||
return False
|
||||
|
||||
|
||||
def sanitize_filename(name: str) -> str:
|
||||
"""将工作表名转换为稳定的文件名片段。"""
|
||||
invalid_chars = '<>:"/\\|?*'
|
||||
sanitized = "".join("_" if ch in invalid_chars else ch for ch in name).strip()
|
||||
return sanitized.rstrip(". ") or "sheet"
|
||||
|
||||
|
||||
def build_text_record(cat_id: int, cat_name: str, row: int, col: int, text: str) -> dict:
|
||||
"""将纯文本单元格兜底转换为 JSONL 记录。"""
|
||||
lines = [line.strip() for line in text.splitlines() if line.strip()]
|
||||
title = lines[0] if lines else text.strip()
|
||||
return {
|
||||
"category_id": cat_id,
|
||||
"category": cat_name,
|
||||
"row": row,
|
||||
"col": col,
|
||||
"title": title[:80],
|
||||
"content": text,
|
||||
}
|
||||
|
||||
|
||||
def run_jsonl_excel_to_jsonl(excel_path: Path, project_root: Path) -> int:
|
||||
"""将内部 JSONL 格式的 Excel 转换为 JSONL 目录（忽略"说明"工作表）"""
|
||||
import json
|
||||
from datetime import datetime
|
||||
try:
|
||||
import pandas as pd
|
||||
except ImportError:
|
||||
print("❌ 需要 pandas: pip install pandas openpyxl")
|
||||
return 1
|
||||
|
||||
xlsx = pd.ExcelFile(excel_path)
|
||||
cat_id = 0
|
||||
total_records = 0
|
||||
written_files = []
|
||||
|
||||
timestamp = datetime.now().strftime("%Y_%m%d_%H%M%S")
|
||||
output_dir = project_root / "prompt_jsonl" / f"{excel_path.stem}_{timestamp}"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
for sheet_index, sheet in enumerate(xlsx.sheet_names, start=1):
|
||||
if sheet == '说明':
|
||||
continue
|
||||
|
||||
cat_id += 1
|
||||
cat_name = sheet
|
||||
df = pd.read_excel(xlsx, sheet_name=sheet, header=None)
|
||||
sheet_lines = []
|
||||
fallback_records = []
|
||||
|
||||
# 检查列名是否是 JSON 数据
|
||||
for col_idx, col_name in enumerate(df.columns):
|
||||
col_str = str(col_name).strip()
|
||||
if col_str.startswith('{') and col_str.endswith('}'):
|
||||
try:
|
||||
obj = json.loads(col_str)
|
||||
if 'title' in obj and 'content' in obj:
|
||||
sheet_lines.append(json.dumps({
|
||||
"category_id": cat_id,
|
||||
"category": cat_name,
|
||||
"row": 1,
|
||||
"col": col_idx + 1,
|
||||
"title": obj["title"][:80],
|
||||
"content": obj["content"]
|
||||
}, ensure_ascii=False))
|
||||
except:
|
||||
pass
|
||||
|
||||
# 处理数据行
|
||||
for row_idx, row in df.iterrows():
|
||||
for col_idx, val in enumerate(row):
|
||||
if pd.isna(val):
|
||||
continue
|
||||
val_str = str(val).strip()
|
||||
if not val_str:
|
||||
continue
|
||||
if val_str.startswith('{') and val_str.endswith('}'):
|
||||
try:
|
||||
obj = json.loads(val_str)
|
||||
if 'title' in obj and 'content' in obj:
|
||||
sheet_lines.append(json.dumps({
|
||||
"category_id": cat_id,
|
||||
"category": cat_name,
|
||||
"row": row_idx + 2,
|
||||
"col": col_idx + 1,
|
||||
"title": obj["title"][:80],
|
||||
"content": obj["content"]
|
||||
}, ensure_ascii=False))
|
||||
except:
|
||||
pass
|
||||
else:
|
||||
# 跳过常见的顶栏广告/元数据噪声，但保留其他纯文本内容作为兜底记录。
|
||||
if row_idx == 0 and col_idx == 0 and val_str.startswith("广告位"):
|
||||
continue
|
||||
fallback_records.append(
|
||||
build_text_record(
|
||||
cat_id=cat_id,
|
||||
cat_name=cat_name,
|
||||
row=row_idx + 2,
|
||||
col=col_idx + 1,
|
||||
text=val_str,
|
||||
)
|
||||
)
|
||||
|
||||
if not sheet_lines and fallback_records:
|
||||
sheet_lines = [
|
||||
json.dumps(record, ensure_ascii=False)
|
||||
for record in fallback_records
|
||||
]
|
||||
|
||||
total_records += len(sheet_lines)
|
||||
file_stem = sanitize_filename(cat_name)
|
||||
output_file = output_dir / f"{sheet_index:02d}_{file_stem}.jsonl"
|
||||
with open(output_file, 'w', encoding='utf-8') as f:
|
||||
if sheet_lines:
|
||||
f.write('\n'.join(sheet_lines) + '\n')
|
||||
written_files.append(output_file)
|
||||
|
||||
if not written_files:
|
||||
print(f"❌ 未找到有效的 JSONL 数据: {excel_path}")
|
||||
return 1
|
||||
|
||||
print(
|
||||
f"✅ Excel(JSONL)→JSONL OK: {excel_path.name} → "
|
||||
f"{output_dir.relative_to(project_root)} "
|
||||
f"({len(written_files)} 个文件 / {total_records} 条记录)"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def run_jsonl_to_excel(jsonl_path: Path, project_root: Path) -> int:
|
||||
"""Convert JSONL to Excel, each cell contains the full JSON object as string."""
|
||||
import json
|
||||
from collections import defaultdict
|
||||
try:
|
||||
import pandas as pd
|
||||
except ImportError:
|
||||
print("❌ 需要 pandas: pip install pandas openpyxl")
|
||||
return 1
|
||||
|
||||
records = []
|
||||
with open(jsonl_path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
if line.strip():
|
||||
records.append(json.loads(line))
|
||||
|
||||
if not records:
|
||||
print(f"❌ JSONL 文件为空: {jsonl_path}")
|
||||
return 1
|
||||
|
||||
# category -> {row -> {col -> json_string}}
|
||||
sheets_data: dict = defaultdict(lambda: defaultdict(dict))
|
||||
cat_id_map = {}
|
||||
|
||||
for r in records:
|
||||
cat_name = r["category"]
|
||||
cat_id_map[r["category_id"]] = cat_name
|
||||
# 单元格内容只保留 title 和 content
|
||||
cell_data = {"title": r["title"], "content": r["content"]}
|
||||
sheets_data[cat_name][r["row"]][r["col"]] = json.dumps(cell_data, ensure_ascii=False)
|
||||
|
||||
output_dir = project_root / "prompt_excel"
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
output_file = output_dir / f"{jsonl_path.stem}.xlsx"
|
||||
|
||||
sorted_cats = sorted(cat_id_map.items(), key=lambda x: x[0])
|
||||
|
||||
with pd.ExcelWriter(output_file, engine='openpyxl') as writer:
|
||||
for cat_id, cat_name in sorted_cats:
|
||||
row_data = sheets_data[cat_name]
|
||||
if not row_data:
|
||||
continue
|
||||
|
||||
max_row = max(row_data.keys())
|
||||
max_col = max(c for cols in row_data.values() for c in cols.keys())
|
||||
|
||||
data = []
|
||||
for row_idx in range(1, max_row + 1):
|
||||
row_list = []
|
||||
for col_idx in range(1, max_col + 1):
|
||||
row_list.append(row_data.get(row_idx, {}).get(col_idx, ""))
|
||||
data.append(row_list)
|
||||
|
||||
df = pd.DataFrame(data)
|
||||
sheet_name = cat_name[:31]
|
||||
df.to_excel(writer, sheet_name=sheet_name, index=False, header=False)
|
||||
|
||||
print(f"✅ JSONL→Excel OK: {jsonl_path.name} → {output_file.relative_to(project_root)} ({len(sorted_cats)} 个工作表)")
|
||||
return 0
|
||||
|
||||
|
||||
def build_candidates(project_root: Path, excel_dir: Path, docs_dir: Path) -> List[Candidate]:
|
||||
candidates: List[Candidate] = []
|
||||
idx = 1
|
||||
jsonl_dir = project_root / "prompt_jsonl"
|
||||
|
||||
for path in list_excel_files(excel_dir):
|
||||
label = f"{path.name}"
|
||||
# 检测是否为内部 JSONL 格式的 Excel
|
||||
if is_jsonl_excel(path):
|
||||
candidates.append(Candidate(index=idx, kind="jsonl_excel", path=path, label=label))
|
||||
else:
|
||||
candidates.append(Candidate(index=idx, kind="excel", path=path, label=label))
|
||||
idx += 1
|
||||
for path in list_doc_sets(docs_dir):
|
||||
display = path.relative_to(project_root) if path.is_absolute() else path
|
||||
# Docs → Excel
|
||||
candidates.append(Candidate(index=idx, kind="docs", path=path, label=f"{display}"))
|
||||
idx += 1
|
||||
# Docs → JSONL
|
||||
candidates.append(Candidate(index=idx, kind="docs2jsonl", path=path, label=f"{display}"))
|
||||
idx += 1
|
||||
for path in list_jsonl_files(jsonl_dir):
|
||||
label = f"{path.name}"
|
||||
candidates.append(Candidate(index=idx, kind="jsonl", path=path, label=label))
|
||||
idx += 1
|
||||
return candidates
|
||||
|
||||
|
||||
def select_interactively(candidates: Sequence[Candidate]) -> Optional[Candidate]:
|
||||
if not candidates:
|
||||
print("没有可用的 Excel 或 Docs 源。请将 .xlsx 放到 prompt_excel/ 或将文档放到 prompt_docs/ 下。")
|
||||
return None
|
||||
|
||||
# Prefer arrow-key selection if available
|
||||
if _INQUIRER_AVAILABLE:
|
||||
try:
|
||||
choices = [
|
||||
{"name": f"[{c.kind.upper()}] {c.label}", "value": c.index}
|
||||
for c in candidates
|
||||
]
|
||||
selection = _inq.select(
|
||||
message="选择要转换的源（上下箭头，回车确认，Ctrl+C 取消）:",
|
||||
choices=choices,
|
||||
default=choices[0]["value"],
|
||||
).execute()
|
||||
match = next((c for c in candidates if c.index == selection), None)
|
||||
return match
|
||||
except KeyboardInterrupt:
|
||||
return None
|
||||
|
||||
if _RICH_AVAILABLE:
|
||||
console = Console()
|
||||
layout = Layout()
|
||||
layout.split_column(
|
||||
Layout(name="header", size=3),
|
||||
Layout(name="list"),
|
||||
Layout(name="footer", size=3),
|
||||
)
|
||||
header = Panel(Text("提示词库转换器", style="bold cyan"), subtitle="选择一个源开始转换", box=box.ROUNDED)
|
||||
|
||||
table = Table(box=box.SIMPLE_HEAVY)
|
||||
table.add_column("编号", style="bold yellow", justify="right", width=4)
|
||||
table.add_column("类型", style="magenta", width=16)
|
||||
table.add_column("路径/名称", style="white")
|
||||
kind_labels = {"excel": "Excel→Docs", "docs": "Docs→Excel", "docs2jsonl": "Docs→JSONL", "jsonl": "JSONL→Excel", "jsonl_excel": "Excel(JSONL)→JSONL"}
|
||||
for c in candidates:
|
||||
table.add_row(str(c.index), kind_labels.get(c.kind, c.kind), c.label)
|
||||
|
||||
layout["header"].update(header)
|
||||
layout["list"].update(Panel(table, title="可选源", border_style="cyan"))
|
||||
layout["footer"].update(Panel(Text("输入编号并回车（0 退出）", style="bold"), box=box.ROUNDED))
|
||||
console.print(layout)
|
||||
|
||||
while True:
|
||||
try:
|
||||
choice = IntPrompt.ask("编号", default=0)
|
||||
except Exception:
|
||||
return None
|
||||
if choice == 0:
|
||||
return None
|
||||
match = next((c for c in candidates if c.index == choice), None)
|
||||
if match is not None:
|
||||
return match
|
||||
console.print("[red]编号不存在，请重试[/red]")
|
||||
|
||||
# Plain fallback
|
||||
kind_labels = {"excel": "Excel→Docs", "docs": "Docs→Excel", "docs2jsonl": "Docs→JSONL", "jsonl": "JSONL→Excel", "jsonl_excel": "Excel(JSONL)→JSONL"}
|
||||
print("请选择一个源进行转换：")
|
||||
for c in candidates:
|
||||
print(f" {c.index:2d}. [{kind_labels.get(c.kind, c.kind)}] {c.label}")
|
||||
print(" 0. 退出")
|
||||
while True:
|
||||
try:
|
||||
raw = input("输入编号后回车：").strip()
|
||||
except EOFError:
|
||||
return None
|
||||
if not raw:
|
||||
continue
|
||||
if raw == "0":
|
||||
return None
|
||||
if not raw.isdigit():
|
||||
print("请输入有效数字。")
|
||||
continue
|
||||
choice = int(raw)
|
||||
match = next((c for c in candidates if c.index == choice), None)
|
||||
if match is None:
|
||||
print("编号不存在，请重试。")
|
||||
continue
|
||||
return match
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
p = argparse.ArgumentParser(description="prompt-library conversion controller")
|
||||
p.add_argument("--excel-dir", type=str, default="prompt_excel", help="Excel sources directory (default: prompt_excel)")
|
||||
p.add_argument("--docs-dir", type=str, default="prompt_docs", help="Docs sources directory (default: prompt_docs)")
|
||||
p.add_argument("--select", type=str, default=None, help="Path to a specific .xlsx file or a docs folder")
|
||||
p.add_argument("--mode", type=str, choices=["excel2docs", "docs2excel", "docs2jsonl", "jsonl2excel", "jsonl_excel2jsonl"], default=None, help="Conversion mode (auto-detect if not specified)")
|
||||
p.add_argument("--non-interactive", action="store_true", help="Do not prompt; require --select or exit")
|
||||
return p.parse_args()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
repo_root = get_repo_root()
|
||||
start_convert = repo_root / "scripts" / "start_convert.py"
|
||||
if not start_convert.exists():
|
||||
print("找不到 scripts/start_convert.py。")
|
||||
return 1
|
||||
|
||||
args = parse_args()
|
||||
|
||||
excel_dir = (repo_root / args.excel_dir).resolve() if not Path(args.excel_dir).is_absolute() else Path(args.excel_dir).resolve()
|
||||
docs_dir = (repo_root / args.docs_dir).resolve() if not Path(args.docs_dir).is_absolute() else Path(args.docs_dir).resolve()
|
||||
|
||||
# Non-interactive path with explicit selection
|
||||
if args.non_interactive or args.select:
|
||||
if not args.select:
|
||||
print("--non-interactive 需要配合 --select 使用。")
|
||||
return 2
|
||||
selected = Path(args.select)
|
||||
if not selected.is_absolute():
|
||||
selected = (repo_root / selected).resolve()
|
||||
if not selected.exists():
|
||||
print(f"选择的路径不存在: {selected}")
|
||||
return 2
|
||||
if selected.is_file() and selected.suffix.lower() == ".xlsx":
|
||||
# 检测是否为内部 JSONL 格式或显式指定模式
|
||||
if args.mode == "jsonl_excel2jsonl" or is_jsonl_excel(selected):
|
||||
return run_jsonl_excel_to_jsonl(selected, repo_root)
|
||||
return run_start_convert(start_convert, mode="excel2docs", project_root=repo_root, select_path=selected, excel_dir=excel_dir)
|
||||
if selected.is_file() and selected.suffix.lower() == ".jsonl":
|
||||
return run_jsonl_to_excel(selected, repo_root)
|
||||
if selected.is_dir():
|
||||
# Check mode or default to docs2excel
|
||||
if args.mode == "docs2jsonl":
|
||||
return run_docs_to_jsonl(selected, repo_root)
|
||||
return run_start_convert(start_convert, mode="docs2excel", project_root=repo_root, select_path=selected, docs_dir=docs_dir)
|
||||
print("无法识别的选择类型。")
|
||||
return 2
|
||||
|
||||
# Interactive selection
|
||||
candidates = build_candidates(repo_root, excel_dir, docs_dir)
|
||||
chosen = select_interactively(candidates)
|
||||
if chosen is None:
|
||||
return 0
|
||||
if chosen.kind == "excel":
|
||||
return run_start_convert(start_convert, mode="excel2docs", project_root=repo_root, select_path=chosen.path, excel_dir=excel_dir)
|
||||
elif chosen.kind == "jsonl_excel":
|
||||
return run_jsonl_excel_to_jsonl(chosen.path, repo_root)
|
||||
elif chosen.kind == "docs2jsonl":
|
||||
return run_docs_to_jsonl(chosen.path, repo_root)
|
||||
elif chosen.kind == "jsonl":
|
||||
return run_jsonl_to_excel(chosen.path, repo_root)
|
||||
else:
|
||||
return run_start_convert(start_convert, mode="docs2excel", project_root=repo_root, select_path=chosen.path, docs_dir=docs_dir)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,6 @@
|
||||
rich
|
||||
InquirerPy
|
||||
pandas
|
||||
openpyxl
|
||||
PyYAML
|
||||
python-dotenv
|
||||
@@ -0,0 +1,90 @@
|
||||
import json
|
||||
import re
|
||||
|
||||
jsonl_file = "prompt_jsonl/prompt_docs_refactored.jsonl"
|
||||
report = []
|
||||
|
||||
def check_md_syntax(text, info_str):
|
||||
lines = text.split('\n')
|
||||
errors = []
|
||||
|
||||
# 1. 检查分隔符 (--- 或 ***)
|
||||
# 规范：应独占一行，前后建议有空行
|
||||
# 正则匹配：行首开始，至少3个-或*，行尾结束，允许行尾有空白
|
||||
separator_pattern = re.compile(r'^\s*([-*]{3,})\s*$')
|
||||
|
||||
# 2. 检查标题 (#)
|
||||
# 规范：#后必须有空格
|
||||
header_pattern = re.compile(r'^(#+)([^ \n].*)') # 捕获 #后紧跟非空格的
|
||||
|
||||
# 3. 代码块 (```)
|
||||
code_block_count = 0
|
||||
|
||||
for i, line in enumerate(lines):
|
||||
# 检查分隔符
|
||||
if separator_pattern.match(line):
|
||||
# 检查长度（虽然md规范>=3即可，但有些习惯是用3个）
|
||||
# 检查前后空行（非强制，但推荐）
|
||||
pass # 暂时只检查基本正则，如果夹杂在文本中通常不会独占一行
|
||||
|
||||
# 检查错误标题： #Title
|
||||
m = header_pattern.match(line)
|
||||
if m:
|
||||
# 排除掉特殊的Shebang或注释，比如 #!/bin/bash 或 #_Role (这个文件里的Title字段用了#_)
|
||||
# 但这里是content字段，应该遵循MD规范
|
||||
# 检查是否在代码块内
|
||||
if code_block_count % 2 == 0:
|
||||
# 忽略一些特定的meta标记，比如 # Role (有些prompt习惯)
|
||||
# 实际上标准MD里 #Role 也是不规范的标题
|
||||
# 允许一些特殊情况? 暂时严格检查
|
||||
errors.append(f"Line {i+1}: 标题格式可能错误 (缺少空格): '{line[:20]}...'" )
|
||||
|
||||
# 检查代码块闭合
|
||||
if line.strip().startswith('```'):
|
||||
code_block_count += 1
|
||||
|
||||
if code_block_count % 2 != 0:
|
||||
errors.append("代码块 (```) 未闭合")
|
||||
|
||||
if errors:
|
||||
report.append(f"\n📄 {info_str}")
|
||||
for e in errors:
|
||||
report.append(f" - {e}")
|
||||
|
||||
def analyze():
|
||||
print("正在检查 Markdown 语法...")
|
||||
try:
|
||||
with open(jsonl_file, 'r', encoding='utf-8') as f:
|
||||
for line_num, line in enumerate(f, 1):
|
||||
if not line.strip(): continue
|
||||
try:
|
||||
item = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
print(f"❌ JSON 解析错误在第 {line_num} 行")
|
||||
continue
|
||||
|
||||
cat = item.get('category', 'Unknown')
|
||||
row = item.get('row', '?')
|
||||
title = item.get('title', 'No Title')
|
||||
content = item.get('content', '')
|
||||
|
||||
if not content:
|
||||
report.append(f"\n⚠️ {cat} | Row {row} | {title}: 内容为空")
|
||||
continue
|
||||
|
||||
info = f"[{cat}] Row {row}: {title}"
|
||||
check_md_syntax(content, info)
|
||||
|
||||
except FileNotFoundError:
|
||||
print("文件未找到")
|
||||
return
|
||||
|
||||
if not report:
|
||||
print("✅ 未发现明显的 Markdown 语法问题。 ")
|
||||
else:
|
||||
print(f"⚠️ 发现潜在问题 ({len(report)} 处):")
|
||||
for msg in report:
|
||||
print(msg)
|
||||
|
||||
if __name__ == "__main__":
|
||||
analyze()
|
||||
@@ -0,0 +1,87 @@
|
||||
# 提示词库同步配置 - 基于Excel完整数据
|
||||
source:
|
||||
excel_file: "prompt (2).xlsx"
|
||||
total_rows: 18
|
||||
total_cols: 3
|
||||
processed_date: "2025-02-02"
|
||||
|
||||
google_sheets:
|
||||
sheet_id: "1ngoQOhJqdguwNAilCl1joNwTje7FWWN9WiI2bo5VhpU"
|
||||
credentials_path: "./credentials.json"
|
||||
|
||||
output:
|
||||
prompts_dir: "./prompts"
|
||||
use_timestamp: true
|
||||
|
||||
naming:
|
||||
max_title_length: 30
|
||||
row_col_format: "({row},{col})"
|
||||
separator: "_"
|
||||
|
||||
sync:
|
||||
skip_rows: [] # 不跳过任何行，完整处理
|
||||
skip_keywords: [] # 完整保留所有内容
|
||||
|
||||
# Excel原始数据映射
|
||||
excel_mapping:
|
||||
prompts:
|
||||
- row: 0
|
||||
title: "提示词1a"
|
||||
versions: [1, 2, 3]
|
||||
content: ["提示词1a", "提示词1b", "提示词1c"]
|
||||
- row: 1
|
||||
title: "提示词2a"
|
||||
versions: [1, 2]
|
||||
content: ["提示词2a", "提示词2b"]
|
||||
- row: 3
|
||||
title: "提示词ya"
|
||||
versions: [1]
|
||||
content: ["提示词ya"]
|
||||
|
||||
tools:
|
||||
openai_optimizer:
|
||||
row: 5
|
||||
url: "https://platform.openai.com/chat/edit?models=gpt-5&optimize=true"
|
||||
description: "openai提示词优化网站"
|
||||
|
||||
social_media:
|
||||
twitter:
|
||||
row: 7
|
||||
url: "https://x.com/123olp"
|
||||
description: "点击关注我的推特，获取最新动态，首页接广告位"
|
||||
|
||||
support:
|
||||
title_row: 9
|
||||
title: "礼貌要饭地址"
|
||||
crypto_wallets:
|
||||
tron:
|
||||
row: 10
|
||||
address: "TQtBXCSTwLFHjBqTS4rNUp7ufiGx51BRey"
|
||||
solana:
|
||||
row: 11
|
||||
address: "HjYhozVf9AQmfv7yv79xSNs6uaEU5oUk2USasYQfUYau"
|
||||
ethereum:
|
||||
row: 12
|
||||
address: "0xa396923a71ee7D9480b346a17dDeEb2c0C287BBC"
|
||||
bsc:
|
||||
row: 13
|
||||
address: "0xa396923a71ee7D9480b346a17dDeEb2c0C287BBC"
|
||||
bitcoin:
|
||||
row: 14
|
||||
address: "bc1plslluj3zq3snpnnczplu7ywf37h89dyudqua04pz4txwh8z5z5vsre7nlm"
|
||||
sui:
|
||||
row: 15
|
||||
address: "0xb720c98a48c77f2d49d375932b2867e793029e6337f1562522640e4f84203d2e"
|
||||
|
||||
misc:
|
||||
warning:
|
||||
row: 17
|
||||
content: "广告位（注意识别风险）"
|
||||
|
||||
# 数据验证规则
|
||||
validation:
|
||||
prompt_rows: [0, 1, 3]
|
||||
tool_rows: [5]
|
||||
social_rows: [7]
|
||||
crypto_rows: [10, 11, 12, 13, 14, 15]
|
||||
warning_rows: [17]
|
||||
@@ -0,0 +1,549 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
convert_local.py
|
||||
|
||||
Reads a local Excel file and converts its contents into a structured prompt library
|
||||
under `prompt-library/` per the development guide. It generates:
|
||||
- prompts/<category>/ (one file per non-empty cell across columns for each prompt row)
|
||||
- prompts/index.json (summary + traceability)
|
||||
- prompts/<category>/index.md (table + version matrix)
|
||||
- docs/tools.md, docs/support.md, docs/excel-data.md
|
||||
- README.md (top-level for prompt-library)
|
||||
|
||||
Usage:
|
||||
python prompt-library/scripts/convert_local.py \
|
||||
[--excel "/absolute/or/relative/path/to/prompt (2).xlsx"] \
|
||||
[--config prompt-library/scripts/config.yaml] \
|
||||
[--category-name prompt-category]
|
||||
|
||||
If no arguments are provided, it will:
|
||||
- load config from prompt-library/scripts/config.yaml (if present)
|
||||
- resolve Excel path from config.source.excel_file relative to project root
|
||||
- default category to "prompt-category"
|
||||
|
||||
Dependencies: pandas, openpyxl, PyYAML
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Tuple
|
||||
|
||||
import pandas as pd
|
||||
|
||||
try:
|
||||
import yaml # type: ignore
|
||||
except Exception: # pragma: no cover
|
||||
yaml = None # Optional; script still works without YAML if no config provided
|
||||
|
||||
|
||||
@dataclass
|
||||
class RowClassification:
|
||||
row_index: int # zero-based excel index
|
||||
kind: str # prompt|tool|social|wallet_header|wallet|warning|other
|
||||
data: Dict
|
||||
|
||||
|
||||
class ExcelPromptConverter:
|
||||
def __init__(
|
||||
self,
|
||||
project_root: Path,
|
||||
prompt_library_dir: Path,
|
||||
excel_path: Path,
|
||||
category_name: str = "prompt-category",
|
||||
config_path: Optional[Path] = None,
|
||||
output_root: Optional[Path] = None,
|
||||
) -> None:
|
||||
self.project_root = project_root
|
||||
self.prompt_library_dir = prompt_library_dir
|
||||
# If an output_root is provided, write into that snapshot directory
|
||||
# rather than the in-repo prompts/docs locations.
|
||||
if output_root is not None:
|
||||
self.output_root = output_root
|
||||
self.prompts_dir = output_root / "prompts"
|
||||
self.docs_dir = output_root / "docs"
|
||||
self.readme_target_root = output_root
|
||||
else:
|
||||
self.output_root = None
|
||||
self.prompts_dir = prompt_library_dir / "prompts"
|
||||
self.docs_dir = prompt_library_dir / "docs"
|
||||
self.readme_target_root = prompt_library_dir
|
||||
self.scripts_dir = prompt_library_dir / "scripts"
|
||||
self.category_name = category_name # fallback if single sheet
|
||||
self.category_dir = self.prompts_dir / self.category_name
|
||||
self.excel_path = excel_path
|
||||
self.config_path = config_path
|
||||
self.config = self._load_config(config_path)
|
||||
self.now = datetime.now()
|
||||
|
||||
# Per-sheet prompts map: {sheet_name: {excel_row -> {title, versions{col->file}}}}
|
||||
self.prompts_info_by_sheet: Dict[str, Dict[int, Dict]] = {}
|
||||
self.tools: List[Dict] = []
|
||||
self.social: List[Dict] = []
|
||||
self.wallets: Dict[str, Dict] = {}
|
||||
self.misc: List[Dict] = []
|
||||
self.total_rows = 0
|
||||
self.total_cols = 0
|
||||
self.sheet_names_order: List[str] = []
|
||||
|
||||
def _load_config(self, config_path: Optional[Path]) -> Dict:
|
||||
if config_path and config_path.exists() and yaml is not None:
|
||||
with config_path.open("r", encoding="utf-8") as f:
|
||||
return yaml.safe_load(f) or {}
|
||||
return {}
|
||||
|
||||
def _sanitize_filename(self, text: str, max_length: int = 60) -> str:
|
||||
if not text:
|
||||
return "untitled"
|
||||
text = str(text).strip()
|
||||
text = re.sub(r"[\\/:*?\"<>|\r\n]+", "", text)
|
||||
text = text.replace(" ", "_")
|
||||
if len(text) > max_length:
|
||||
text = text[:max_length].rstrip("_-")
|
||||
return text or "untitled"
|
||||
|
||||
def _extract_title(self, contents: List[str]) -> str:
|
||||
for c in contents:
|
||||
if c and c.strip():
|
||||
first_line = c.strip().splitlines()[0]
|
||||
words = first_line.split()
|
||||
candidate = " ".join(words[:6])
|
||||
return self._sanitize_filename(candidate)
|
||||
return "untitled"
|
||||
|
||||
def _read_excel_sheets(self) -> Dict[str, pd.DataFrame]:
|
||||
# Read all sheets; if workbook has single sheet, still returns dict with one entry
|
||||
sheets: Dict[str, pd.DataFrame] = pd.read_excel(self.excel_path, header=None, engine="openpyxl", sheet_name=None) # type: ignore
|
||||
normalized: Dict[str, pd.DataFrame] = {}
|
||||
for sheet_name, df in sheets.items():
|
||||
try:
|
||||
df = df.map(lambda v: v.strip() if isinstance(v, str) else v) # pandas >=2.1
|
||||
except Exception:
|
||||
df = df.applymap(lambda v: v.strip() if isinstance(v, str) else v) # fallback
|
||||
normalized[sheet_name] = df
|
||||
# preserve order of sheets
|
||||
self.sheet_names_order = list(normalized.keys())
|
||||
# set global rows/cols to first sheet for summary; detailed per-sheet handled later
|
||||
if normalized:
|
||||
any_df = normalized[self.sheet_names_order[0]]
|
||||
self.total_rows, self.total_cols = any_df.shape
|
||||
return normalized
|
||||
|
||||
def _classify_rows(self, df: pd.DataFrame) -> List[RowClassification]:
|
||||
classifications: List[RowClassification] = []
|
||||
wallet_mode = False
|
||||
|
||||
for r in range(df.shape[0]):
|
||||
row_vals = [df.iloc[r, c] if c < df.shape[1] else None for c in range(df.shape[1])]
|
||||
non_empty = [v for v in row_vals if isinstance(v, str) and v.strip()]
|
||||
any_http = any(isinstance(v, str) and v.startswith("http") for v in row_vals)
|
||||
|
||||
if not non_empty:
|
||||
classifications.append(RowClassification(r, "other", {"empty": True}))
|
||||
continue
|
||||
|
||||
# Wallet header detection (e.g., contains "网络" and a label like "礼貌要饭地址")
|
||||
joined = " ".join([v for v in non_empty])
|
||||
if any(k in joined for k in ["网络", "网络名称"]) and any(
|
||||
k in joined for k in ["礼貌要饭地址", "钱包", "地址"]
|
||||
):
|
||||
wallet_mode = True
|
||||
classifications.append(RowClassification(r, "wallet_header", {"raw": row_vals}))
|
||||
continue
|
||||
|
||||
if wallet_mode:
|
||||
# If the row still looks like wallet data (two columns: network, address)
|
||||
first, second = row_vals[0] if len(row_vals) > 0 else None, row_vals[1] if len(row_vals) > 1 else None
|
||||
if (first and isinstance(first, str)) and (second and isinstance(second, str)):
|
||||
classifications.append(
|
||||
RowClassification(
|
||||
r,
|
||||
"wallet",
|
||||
{
|
||||
"network": first,
|
||||
"address": second,
|
||||
"raw": row_vals,
|
||||
},
|
||||
)
|
||||
)
|
||||
continue
|
||||
else:
|
||||
wallet_mode = False # end wallet section if pattern breaks
|
||||
|
||||
# Tools and social heuristics
|
||||
if any_http:
|
||||
url = next(v for v in row_vals if isinstance(v, str) and v.startswith("http"))
|
||||
desc = None
|
||||
for v in row_vals:
|
||||
if v and isinstance(v, str) and not v.startswith("http"):
|
||||
desc = v
|
||||
break
|
||||
kind = "social" if ("x.com" in url or "twitter.com" in url) else "tool"
|
||||
classifications.append(RowClassification(r, kind, {"url": url, "description": desc or "", "raw": row_vals}))
|
||||
continue
|
||||
|
||||
# Warnings or misc markers
|
||||
if any("广告位" in v for v in non_empty if isinstance(v, str)):
|
||||
classifications.append(RowClassification(r, "warning", {"content": joined, "raw": row_vals}))
|
||||
continue
|
||||
|
||||
# Placeholder rows to ignore as prompts
|
||||
if any(v in {"...", "….", "...."} for v in non_empty):
|
||||
classifications.append(RowClassification(r, "other", {"placeholder": True, "raw": row_vals}))
|
||||
continue
|
||||
|
||||
# Otherwise: treat as prompt row (one logical prompt per row with multiple versions across columns)
|
||||
prompt_versions: Dict[int, str] = {}
|
||||
for c in range(df.shape[1]):
|
||||
cell = df.iloc[r, c] if c < df.shape[1] else None
|
||||
if isinstance(cell, str) and cell.strip():
|
||||
prompt_versions[c + 1] = cell.strip()
|
||||
if prompt_versions:
|
||||
classifications.append(RowClassification(r, "prompt", {"versions": prompt_versions}))
|
||||
else:
|
||||
classifications.append(RowClassification(r, "other", {"raw": row_vals}))
|
||||
|
||||
return classifications
|
||||
|
||||
def _ensure_dirs(self) -> None:
|
||||
self.prompts_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.category_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.docs_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def _write_prompt_file(self, row_num: int, col_num: int, title: str, content: str, versions_in_row: List[int]) -> str:
|
||||
"""Write a prompt file containing ONLY the prompt text, nothing else."""
|
||||
row_col = f"({row_num},{col_num})"
|
||||
filename = f"{row_col}_{title}.md"
|
||||
filepath = self.category_dir / filename
|
||||
# Ensure content ends with newline and contains no surrounding fences/headers added by us
|
||||
pure = (content or "").rstrip("\n") + "\n"
|
||||
filepath.write_text(pure, encoding="utf-8")
|
||||
return filename
|
||||
|
||||
def _generate_category_index(self, sheet_name: str, category_dir: Path, prompts_info: Dict[int, Dict]) -> None:
|
||||
index_path = category_dir / "index.md"
|
||||
total_prompts = len(prompts_info)
|
||||
total_versions = sum(len(meta["versions"]) for meta in prompts_info.values())
|
||||
avg_versions = total_versions / total_prompts if total_prompts else 0
|
||||
|
||||
lines: List[str] = []
|
||||
lines.append(f"# 📂 提示词分类 - {sheet_name}（基于Excel原始数据)\n")
|
||||
lines.append(f"最后同步: {self.now.strftime('%Y-%m-%d %H:%M:%S')}\n")
|
||||
lines.append("\n## 📊 统计\n")
|
||||
lines.append(f"- 提示词总数: {total_prompts}\n")
|
||||
lines.append(f"- 版本总数: {total_versions} \n")
|
||||
lines.append(f"- 平均版本数: {avg_versions:.1f}\n\n")
|
||||
lines.append("## 📋 提示词列表\n")
|
||||
lines.append("\n| 序号 | 标题 | 版本数 | 查看 |\n|------|------|--------|------|\n")
|
||||
for row in sorted(prompts_info.keys()):
|
||||
info = prompts_info[row]
|
||||
title = info["title"]
|
||||
versions = info["versions"]
|
||||
links = " / ".join([f"[v{v}](./({row},{v})_{title}.md)" for v in sorted(versions.keys())])
|
||||
lines.append(f"| {row} | {title} | {len(versions)} | {links} |\n")
|
||||
|
||||
# Version matrix
|
||||
max_col = 0
|
||||
for info in prompts_info.values():
|
||||
if info["versions"]:
|
||||
max_col = max(max_col, max(info["versions"].keys()))
|
||||
lines.append("\n## 🗂️ 版本矩阵\n")
|
||||
header = ["行"] + [f"v{i}" for i in range(1, max_col + 1)] + ["备注"]
|
||||
lines.append("\n| " + " | ".join(header) + " |\n" + "|" + "---|" * len(header) + "\n")
|
||||
for row in sorted(prompts_info.keys()):
|
||||
info = prompts_info[row]
|
||||
row_cells = [str(row)]
|
||||
for c in range(1, max_col + 1):
|
||||
row_cells.append("✅" if c in info["versions"] else "—")
|
||||
row_cells.append("")
|
||||
lines.append("| " + " | ".join(row_cells) + " |\n")
|
||||
|
||||
index_path.write_text("\n".join(lines), encoding="utf-8")
|
||||
|
||||
def _generate_prompts_index_json(self) -> None:
|
||||
index_json_path = self.prompts_dir / "index.json"
|
||||
total_prompts = sum(len(p) for p in self.prompts_info_by_sheet.values())
|
||||
total_versions = sum(sum(len(meta["versions"]) for meta in p.values()) for p in self.prompts_info_by_sheet.values())
|
||||
stats = {
|
||||
"sheets": len(self.prompts_info_by_sheet),
|
||||
"prompts": total_prompts,
|
||||
"versions": total_versions,
|
||||
"tools": len(self.tools) if self.tools else 0,
|
||||
"social_accounts": len(self.social) if self.social else 0,
|
||||
"crypto_wallets": len(self.wallets) if self.wallets else 0,
|
||||
}
|
||||
categories = []
|
||||
for sheet_name in self.sheet_names_order:
|
||||
prompts_info = self.prompts_info_by_sheet.get(sheet_name, {})
|
||||
categories.append(
|
||||
{
|
||||
"name": sheet_name,
|
||||
"prompt_count": len(prompts_info),
|
||||
"version_count": sum(len(meta["versions"]) for meta in prompts_info.values()),
|
||||
"prompts": [
|
||||
{
|
||||
"row": row,
|
||||
"title": info["title"],
|
||||
"versions": sorted(list(info["versions"].keys())),
|
||||
"files": [info["versions"][v] for v in sorted(info["versions"].keys())],
|
||||
}
|
||||
for row, info in sorted(prompts_info.items())
|
||||
],
|
||||
}
|
||||
)
|
||||
excel_data = {
|
||||
"total_rows": self.total_rows,
|
||||
"total_cols": self.total_cols,
|
||||
"sheets": list(self.prompts_info_by_sheet.keys()),
|
||||
}
|
||||
tools = {}
|
||||
if self.tools:
|
||||
for t in self.tools:
|
||||
name = t.get("name") or "tool"
|
||||
tools[name] = {k: v for k, v in t.items() if k != "name"}
|
||||
social_media = {}
|
||||
if self.social:
|
||||
for s in self.social:
|
||||
name = s.get("name") or "social"
|
||||
social_media[name] = {k: v for k, v in s.items() if k != "name"}
|
||||
support = {
|
||||
"description": "礼貌要饭地址",
|
||||
"crypto_wallets": self.wallets,
|
||||
}
|
||||
data = {
|
||||
"last_updated": self.now.strftime("%Y-%m-%dT%H:%M:%S"),
|
||||
"source": self.excel_path.name,
|
||||
"stats": stats,
|
||||
"categories": categories,
|
||||
"excel_data": excel_data,
|
||||
"tools": tools,
|
||||
"social_media": social_media,
|
||||
"support": support,
|
||||
"misc": self.misc,
|
||||
}
|
||||
index_json_path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
def _generate_docs(self, sheets: Dict[str, pd.DataFrame]) -> None:
|
||||
# docs/excel-data.md (full table)
|
||||
excel_doc_path = self.docs_dir / "excel-data.md"
|
||||
lines: List[str] = []
|
||||
lines.append("# 📊 Excel原始数据完整记录\n")
|
||||
lines.append("## 数据来源\n")
|
||||
lines.append(f"- **文件**: {self.excel_path.name}\n")
|
||||
lines.append(f"- **处理时间**: {self.now.strftime('%Y-%m-%d')}\n")
|
||||
lines.append(f"- **工作表数量**: {len(sheets)}\n\n")
|
||||
for sheet_name, df in sheets.items():
|
||||
rows, cols = df.shape
|
||||
lines.append(f"## 工作表: {sheet_name} ({rows}行×{cols}列)\n")
|
||||
lines.append("\n| 行号 | 列1 | 列2 | 列3 |\n|-----:|-----|-----|-----|\n")
|
||||
for r in range(rows):
|
||||
c1 = df.iloc[r, 0] if cols > 0 else ""
|
||||
c2 = df.iloc[r, 1] if cols > 1 else ""
|
||||
c3 = df.iloc[r, 2] if cols > 2 else ""
|
||||
def fmt(x) -> str:
|
||||
try:
|
||||
if x is None or (isinstance(x, float) and pd.isna(x)) or (hasattr(pd, 'isna') and pd.isna(x)):
|
||||
return ""
|
||||
except Exception:
|
||||
pass
|
||||
s = str(x)
|
||||
return s.replace("|", "\\|")
|
||||
lines.append(f"| {r} | {fmt(c1)} | {fmt(c2)} | {fmt(c3)} |\n")
|
||||
lines.append("\n")
|
||||
lines.append("\n---\n*完整数据提取自 {0}*\n".format(self.excel_path.name))
|
||||
excel_doc_path.write_text("\n".join(lines), encoding="utf-8")
|
||||
|
||||
# docs/tools.md
|
||||
tools_path = self.docs_dir / "tools.md"
|
||||
t_lines: List[str] = []
|
||||
t_lines.append("# 🛠️ 工具与资源（从Excel提取）\n")
|
||||
if self.tools:
|
||||
t_lines.append("\n## AI优化工具\n")
|
||||
for t in self.tools:
|
||||
t_lines.append("\n### {0}\n- **URL**: {1}\n- **描述**: {2}\n- **数据来源**: Excel表格第{3}行\n".format(
|
||||
t.get("name") or "工具",
|
||||
t.get("url", ""),
|
||||
t.get("description", ""),
|
||||
(t.get("excel_row") or 0) + 1,
|
||||
))
|
||||
if self.social:
|
||||
t_lines.append("\n## 社交媒体\n")
|
||||
for s in self.social:
|
||||
t_lines.append("\n### {0}\n- **URL**: {1}\n- **描述**: {2}\n- **数据来源**: Excel表格第{3}行\n".format(
|
||||
s.get("name") or "社交账号",
|
||||
s.get("url", ""),
|
||||
s.get("description", ""),
|
||||
(s.get("excel_row") or 0) + 1,
|
||||
))
|
||||
t_lines.append("\n## 使用建议\n\n1. **OpenAI优化器**: 可以用来测试和改进本库中的提示词\n2. **社交媒体**: 关注获取项目更新和使用技巧\n3. **集成方式**: 可以将这些工具集成到自动化工作流中\n\n---\n*数据来源: {0}*\n".format(self.excel_path.name))
|
||||
tools_path.write_text("\n".join(t_lines), encoding="utf-8")
|
||||
|
||||
# docs/support.md
|
||||
support_path = self.docs_dir / "support.md"
|
||||
s_lines: List[str] = []
|
||||
s_lines.append("# 💰 项目支持（从Excel提取）\n")
|
||||
s_lines.append("\n## 支持说明\n**礼貌要饭地址** - 如果这个项目对您有帮助，欢迎通过以下方式支持\n")
|
||||
if self.wallets:
|
||||
s_lines.append("\n## 加密货币钱包地址\n\n### 主流网络支持\n")
|
||||
s_lines.append("\n| 网络名称 | 钱包地址 | Excel行号 |\n|----------|----------|-----------|\n")
|
||||
for net, data in self.wallets.items():
|
||||
s_lines.append("| **{0}** | `{1}` | 第{2}行 |\n".format(net.upper(), data.get("address", ""), (data.get("excel_row") or 0) + 1))
|
||||
if self.misc:
|
||||
for m in self.misc:
|
||||
if m.get("type") == "warning" or "广告位" in m.get("content", ""):
|
||||
s_lines.append("\n⚠️ **重要提醒**: {0}\n".format(m.get("content")))
|
||||
s_lines.append("\n### 使用建议\n1. 请确认钱包地址的准确性\n2. 建议小额测试后再进行大额转账\n3. 不同网络的转账费用不同，请选择合适的网络\n\n---\n*钱包地址来源: {0}*\n".format(self.excel_path.name))
|
||||
support_path.write_text("\n".join(s_lines), encoding="utf-8")
|
||||
|
||||
def _generate_readme(self) -> None:
|
||||
readme_path = self.readme_target_root / "README.md"
|
||||
total_prompts = sum(len(p) for p in self.prompts_info_by_sheet.values())
|
||||
total_versions = sum(sum(len(meta["versions"]) for meta in p.values()) for p in self.prompts_info_by_sheet.values())
|
||||
readme = []
|
||||
readme.append("# 📚 提示词库（Excel转换版）\n")
|
||||
readme.append("")
|
||||
readme.append(f"")
|
||||
readme.append(f"")
|
||||
readme.append(f"\n")
|
||||
readme.append(f"最后更新: {self.now.strftime('%Y-%m-%d %H:%M:%S')}\n")
|
||||
readme.append("\n## 📊 总览\n")
|
||||
readme.append(f"- **数据来源**: {self.excel_path.name}\n")
|
||||
readme.append(f"- **分类数量**: {len(self.prompts_info_by_sheet)} \n- **提示词总数**: {total_prompts}\n- **版本总数**: {total_versions}\n")
|
||||
readme.append("\n## 📂 分类导航\n")
|
||||
for i, sheet_name in enumerate(self.sheet_names_order, start=1):
|
||||
prompts_info = self.prompts_info_by_sheet.get(sheet_name, {})
|
||||
folder = f"({i})_{self._sanitize_filename(sheet_name)}"
|
||||
ver_count = sum(len(meta["versions"]) for meta in prompts_info.values())
|
||||
readme.append(f"- [{sheet_name}](./prompts/{folder}/) - {len(prompts_info)} 个提示词, {ver_count} 个版本\n")
|
||||
readme.append("\n## 🔄 同步信息\n")
|
||||
readme.append(f"- **数据源**: {self.excel_path.name}\n- **处理时间**: {self.now.strftime('%Y-%m-%d %H:%M:%S')}\n")
|
||||
readme.append("\n## 📝 许可证\n本项目采用 MIT 许可证\n")
|
||||
readme.append("\n---\n*完全基于 Excel 表格自动生成*\n")
|
||||
readme_path.write_text("\n".join(readme), encoding="utf-8")
|
||||
|
||||
def convert(self) -> None:
|
||||
self._ensure_dirs()
|
||||
sheets = self._read_excel_sheets()
|
||||
# If no sheets returned (shouldn't happen), fallback to empty
|
||||
for idx, sheet_name in enumerate(self.sheet_names_order, start=1):
|
||||
df = sheets[sheet_name]
|
||||
# Prepare per-sheet folder
|
||||
folder_name = f"({idx})_{self._sanitize_filename(sheet_name)}"
|
||||
category_dir = self.prompts_dir / folder_name
|
||||
category_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Classify rows
|
||||
rows = self._classify_rows(df)
|
||||
prompts_info: Dict[int, Dict] = {}
|
||||
|
||||
# Build prompt files for this sheet
|
||||
for rc in rows:
|
||||
if rc.kind == "prompt":
|
||||
excel_row_number = rc.row_index + 1
|
||||
versions: Dict[int, str] = rc.data["versions"]
|
||||
title = self._extract_title(list(versions.values()))
|
||||
prompts_info[excel_row_number] = {"title": title, "versions": {}}
|
||||
# Rewrite files directly into category_dir
|
||||
for col_num, content in versions.items():
|
||||
row_col = f"({excel_row_number},{col_num})"
|
||||
filename = f"{row_col}_{title}.md"
|
||||
(category_dir / filename).write_text((content or "").rstrip("\n") + "\n", encoding="utf-8")
|
||||
prompts_info[excel_row_number]["versions"][col_num] = filename
|
||||
elif rc.kind == "tool":
|
||||
url = rc.data.get("url", "")
|
||||
self.tools.append({
|
||||
"name": "OpenAI 提示词优化平台" if "openai" in url else "工具",
|
||||
"url": url,
|
||||
"description": rc.data.get("description", ""),
|
||||
"excel_row": rc.row_index,
|
||||
"sheet": sheet_name,
|
||||
})
|
||||
elif rc.kind == "social":
|
||||
url = rc.data.get("url", "")
|
||||
name = "Twitter/X 账号" if ("x.com" in url or "twitter.com" in url) else "社交账号"
|
||||
self.social.append({
|
||||
"name": name,
|
||||
"url": url,
|
||||
"description": rc.data.get("description", ""),
|
||||
"excel_row": rc.row_index,
|
||||
"sheet": sheet_name,
|
||||
})
|
||||
elif rc.kind == "wallet":
|
||||
network = str(rc.data.get("network", "")).strip()
|
||||
address = str(rc.data.get("address", "")).strip()
|
||||
if network and address:
|
||||
self.wallets[network.lower()] = {
|
||||
"address": address,
|
||||
"excel_row": rc.row_index,
|
||||
"sheet": sheet_name,
|
||||
}
|
||||
elif rc.kind == "warning":
|
||||
self.misc.append({"type": "warning", "excel_row": rc.row_index, "content": rc.data.get("content", ""), "sheet": sheet_name})
|
||||
|
||||
# Save per-sheet prompts map and index
|
||||
self.prompts_info_by_sheet[sheet_name] = prompts_info
|
||||
self._generate_category_index(sheet_name, category_dir, prompts_info)
|
||||
|
||||
# Global indices and docs
|
||||
self._generate_prompts_index_json()
|
||||
self._generate_docs(sheets)
|
||||
self._generate_readme()
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description="Convert local Excel into prompt library structure")
|
||||
parser.add_argument("--excel", type=str, default=None, help="Path to the Excel file (default from config)")
|
||||
parser.add_argument("--config", type=str, default=None, help="Path to config.yaml (optional)")
|
||||
parser.add_argument("--category-name", type=str, default="prompt-category", help="Output category folder name")
|
||||
parser.add_argument("--out-dir", type=str, default=None, help="Optional snapshot output root. If set, writes to <out-dir>/prompts and <out-dir>/docs")
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
|
||||
script_path = Path(__file__).resolve()
|
||||
prompt_library_dir = script_path.parent.parent
|
||||
project_root = prompt_library_dir.parent
|
||||
|
||||
config_path = Path(args.config).resolve() if args.config else (prompt_library_dir / "scripts" / "config.yaml")
|
||||
|
||||
# Resolve Excel path
|
||||
if args.excel:
|
||||
excel_path = Path(args.excel)
|
||||
if not excel_path.is_absolute():
|
||||
excel_path = (project_root / excel_path).resolve()
|
||||
else:
|
||||
# Try config
|
||||
cfg_excel = None
|
||||
if config_path.exists() and yaml is not None:
|
||||
with config_path.open("r", encoding="utf-8") as f:
|
||||
cfg = yaml.safe_load(f) or {}
|
||||
cfg_excel = ((cfg.get("source") or {}).get("excel_file") or None)
|
||||
excel_path = (project_root / cfg_excel).resolve() if cfg_excel else (project_root / "prompt (2).xlsx").resolve()
|
||||
|
||||
if not excel_path.exists():
|
||||
raise FileNotFoundError(f"Excel file not found: {excel_path}")
|
||||
|
||||
out_dir = Path(args.out_dir).resolve() if args.out_dir else None
|
||||
|
||||
converter = ExcelPromptConverter(
|
||||
project_root=project_root,
|
||||
prompt_library_dir=prompt_library_dir,
|
||||
excel_path=excel_path,
|
||||
category_name=args.category_name,
|
||||
config_path=config_path if config_path.exists() else None,
|
||||
output_root=out_dir,
|
||||
)
|
||||
converter.convert()
|
||||
target = out_dir if out_dir else prompt_library_dir
|
||||
print(f"✅ Conversion complete. Output under: {target}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,118 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
docs_to_excel.py
|
||||
|
||||
Documents → Excel converter: rebuild a workbook from prompts folders.
|
||||
|
||||
Rules (per STRUCTURE_AND_CONVERSION_SPEC.md):
|
||||
- Each folder under prompt-library/prompts that matches "(N)_<name>" or any folder is a sheet
|
||||
- For each file matching "(r,c)_*.md", write its full text to Excel cell (r,c), 1-based
|
||||
- Title part in filename is ignored for cell value
|
||||
- Non-matching files are ignored
|
||||
- Optionally clears existing workbook or merges (default: overwrite generate new)
|
||||
|
||||
Usage:
|
||||
python prompt-library/scripts/docs_to_excel.py --out "rebuilt.xlsx"
|
||||
# optional: --prompts-dir prompt-library/prompts --clear
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Dict, Tuple
|
||||
|
||||
import pandas as pd
|
||||
from openpyxl import Workbook
|
||||
|
||||
FOLDER_PREFIX_RE = re.compile(r"^\((\d+)\)_")
|
||||
FILE_NAME_RE = re.compile(r"^\((\d+),(\d+)\)_.*\.md$")
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
p = argparse.ArgumentParser(description="Rebuild Excel workbook from prompt folders")
|
||||
p.add_argument("--prompts-dir", type=str, default="prompt-library/prompts", help="Prompts root directory")
|
||||
p.add_argument("--out", type=str, required=True, help="Output Excel file path")
|
||||
return p.parse_args()
|
||||
|
||||
|
||||
def list_sheet_folders(prompts_root: Path) -> Dict[str, Path]:
|
||||
sheets: Dict[str, Path] = {}
|
||||
for child in sorted(prompts_root.iterdir()):
|
||||
if not child.is_dir():
|
||||
continue
|
||||
if child.name == "prompt-category":
|
||||
# legacy; skip auto-generated category
|
||||
continue
|
||||
sheets[child.name] = child
|
||||
return sheets
|
||||
|
||||
|
||||
def extract_rc(name: str) -> Tuple[int, int] | None:
|
||||
m = FILE_NAME_RE.match(name)
|
||||
if not m:
|
||||
return None
|
||||
r = int(m.group(1))
|
||||
c = int(m.group(2))
|
||||
return r, c
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
prompts_root = Path(args.prompts_dir).resolve()
|
||||
out_path = Path(args.out).resolve()
|
||||
|
||||
if not prompts_root.exists():
|
||||
raise FileNotFoundError(f"Prompts directory not found: {prompts_root}")
|
||||
|
||||
sheet_folders = list_sheet_folders(prompts_root)
|
||||
if not sheet_folders:
|
||||
raise RuntimeError("No sheet folders found under prompts root")
|
||||
|
||||
wb = Workbook()
|
||||
# remove default sheet
|
||||
default = wb.active
|
||||
wb.remove(default)
|
||||
|
||||
for folder_name, folder_path in sheet_folders.items():
|
||||
# Recover original sheet name (try to drop ordering prefix "(N)_")
|
||||
m = FOLDER_PREFIX_RE.match(folder_name)
|
||||
sheet_name = folder_name[m.end():] if m else folder_name
|
||||
if not sheet_name:
|
||||
sheet_name = folder_name
|
||||
ws = wb.create_sheet(title=sheet_name)
|
||||
|
||||
# Aggregate cells
|
||||
max_row = 0
|
||||
max_col = 0
|
||||
cells: Dict[Tuple[int, int], str] = {}
|
||||
for file in folder_path.iterdir():
|
||||
if not file.is_file() or not file.name.endswith('.md'):
|
||||
continue
|
||||
rc = extract_rc(file.name)
|
||||
if not rc:
|
||||
continue
|
||||
r, c = rc
|
||||
text = file.read_text(encoding='utf-8')
|
||||
# Trim a single trailing newline for cell value aesthetics
|
||||
if text.endswith("\n"):
|
||||
text = text[:-1]
|
||||
cells[(r, c)] = text
|
||||
if r > max_row:
|
||||
max_row = r
|
||||
if c > max_col:
|
||||
max_col = c
|
||||
|
||||
# Write into sheet
|
||||
for (r, c), val in cells.items():
|
||||
ws.cell(row=r, column=c, value=val)
|
||||
|
||||
# Save workbook
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
wb.save(str(out_path))
|
||||
print(f"✅ Rebuilt Excel saved to: {out_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,33 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
excel_to_docs.py
|
||||
|
||||
Thin wrapper that invokes the Excel → Documents converter implemented
|
||||
in convert_local.py, keeping a clearer entrypoint name.
|
||||
|
||||
Usage:
|
||||
python prompt-library/scripts/excel_to_docs.py --excel "prompt (2).xlsx"
|
||||
# optional:
|
||||
# --category-name <fallback> --config prompt-library/scripts/config.yaml
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import importlib.util
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main() -> None:
|
||||
script = Path(__file__).resolve().parent / "convert_local.py"
|
||||
spec = importlib.util.spec_from_file_location("convert_local", str(script))
|
||||
if spec is None or spec.loader is None:
|
||||
raise RuntimeError("Unable to load convert_local.py")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules["convert_local"] = module
|
||||
spec.loader.exec_module(module) # type: ignore
|
||||
# Delegate to its CLI
|
||||
module.main() # type: ignore
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,53 @@
|
||||
import json
|
||||
import shutil
|
||||
from collections import defaultdict
|
||||
|
||||
input_file = "prompt_jsonl/prompt_docs_refactored.jsonl"
|
||||
output_file = "prompt_jsonl/prompt_docs_refactored_clean.jsonl"
|
||||
backup_file = "prompt_jsonl/prompt_docs_refactored.jsonl.bak"
|
||||
|
||||
def filter_versions():
|
||||
# 1. Backup
|
||||
shutil.copy(input_file, backup_file)
|
||||
print(f"Backup created: {backup_file}")
|
||||
|
||||
# 2. Group by (category, row) and find max col
|
||||
latest_versions = {} # Key: (category, row), Value: item_dict
|
||||
|
||||
with open(input_file, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
if not line.strip(): continue
|
||||
item = json.loads(line)
|
||||
|
||||
cat = item.get('category', 'Uncategorized')
|
||||
row = item.get('row', 0)
|
||||
col = item.get('col', 0)
|
||||
|
||||
key = (cat, row)
|
||||
|
||||
if key not in latest_versions:
|
||||
latest_versions[key] = item
|
||||
else:
|
||||
# If current item has higher col, replace it
|
||||
if col > latest_versions[key].get('col', 0):
|
||||
latest_versions[key] = item
|
||||
|
||||
# 3. Write filtered data
|
||||
count = 0
|
||||
with open(output_file, 'w', encoding='utf-8') as f:
|
||||
# Sort by category then row for tidiness
|
||||
sorted_keys = sorted(latest_versions.keys(), key=lambda x: (x[0], x[1]))
|
||||
for key in sorted_keys:
|
||||
item = latest_versions[key]
|
||||
f.write(json.dumps(item, ensure_ascii=False) + '\n')
|
||||
count += 1
|
||||
|
||||
print(f"Filtered file written: {output_file}")
|
||||
print(f"Total prompts retained: {count}")
|
||||
|
||||
# Overwrite original for downstream scripts
|
||||
shutil.move(output_file, input_file)
|
||||
print(f"Overwritten original file: {input_file}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
filter_versions()
|
||||
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
使用 Gemini CLI 按固定系统提示词，将指定目录下的 .md 提示词批量转换为 JSONL。
|
||||
|
||||
特点：
|
||||
- 内置系统提示词，与《Gemini 无头模式 JSONL 规范化指引》一致
|
||||
- 禁用工具调用 (--allowed-tools ''), 输出纯文本，每个文件生成一行 JSON
|
||||
- 默认输入目录为仓库根下的 `2/`，输出为 `2/prompts.jsonl`
|
||||
|
||||
用法示例：
|
||||
python3 gemini_jsonl_batch.py
|
||||
python3 gemini_jsonl_batch.py --input 2 --output 2/prompts.jsonl --model gemini-2.5-flash
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# ==================== 固定系统提示词 ====================
|
||||
SYS_PROMPT = """{"category_id": 1, "category": "JSONL规范化", "row": 2, "col": 1, "title": "# JSONL 提示词转换器 - 系统提示词", "content": "# JSONL 提示词转换器 - 系统提示词\n\n你是一个专业 的提示词格式转换器。将用户提供的提示词内容转换为标准 JSONL 格式。\n\n## 输出格式\n\n```json\n{\\"title\\\": \\"<标题>\\", \\"content\\\": \\"<完整内容>\\"}\n```\n\n### 字段说明\n\n| 字段 | 类型 | 说明 |\n|------|------|------|\n| `title` | string | 提示词标题，取内容的第一行或前 50 字符 |\n| `content` | string | 完整的提示词内容 |\n\n## 转换规则\n\n1. **标题提取**：\n - 若内容以 `#` 开头，取第一个标题作为 title\n - 否则取前 50 字符（去除换行）\n2. **内容转义**：\n - 换行符 转为 `\\\\n`\n - 双引号转为 `\\\\\"`\n - 反斜杠转为 `\\\\\\\\`\n\n## 输出要求\n\n- 每行一个完整的 JSON 对象\n- 不要添加任何解释、注释或额外文字\n- 不要用 ```json 代码块包裹\n- 直接输出纯 JSONL 内容\n\n## 示例\n\n### 输入\n```\n# Role：智能文档助手\n\n## Background\n用户需要一个能够处理文档的 AI 助手。\n\n## Skills\n- 文档解析\n- 格式转换\n```\n\n### 输出\n```\n{\\"title\\\": \\"# Role：智能文档助手\\", \\"content\\\": \\"# Role：智能文档助手\\\\n\\\\n## Background\\\\n用户需要一个能够处 理文档的 AI 助手。\\\\n\\\\n## Skills\\\\n- 文档解析\\\\n- 格式转换\\"}\n```\n\n---\n\n现在，请将用户提供的内容转换为标准 JSONL 格式。"}"""
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="使用 Gemini CLI 批量将 .md 提示词转换为 JSONL（固定系统提示词）。",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-i",
|
||||
"--input",
|
||||
type=Path,
|
||||
default=Path("2"),
|
||||
help="输入目录，遍历其中的 .md 文件（默认：仓库根目录下的 2/）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-o",
|
||||
"--output",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="输出 JSONL 文件路径，默认写入 <input>/prompts.jsonl",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-m",
|
||||
"--model",
|
||||
default="gemini-2.5-flash",
|
||||
help="Gemini 模型名称（默认：gemini-2.5-flash）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--gemini-cmd",
|
||||
default="gemini",
|
||||
help="Gemini CLI 可执行文件名或路径（默认：gemini）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-v",
|
||||
"--verbose",
|
||||
action="store_true",
|
||||
help="输出处理中的详细信息",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def run_gemini(content: str, model: str, cmd: str) -> str:
|
||||
"""调用 Gemini CLI，将单个文本转换为一行 JSON。"""
|
||||
proc = subprocess.run(
|
||||
[
|
||||
cmd,
|
||||
"-m",
|
||||
model,
|
||||
"--output-format",
|
||||
"text",
|
||||
"--allowed-tools",
|
||||
"",
|
||||
SYS_PROMPT,
|
||||
],
|
||||
input=content.encode("utf-8"),
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
check=False,
|
||||
)
|
||||
|
||||
stdout = proc.stdout.decode("utf-8", errors="replace").strip()
|
||||
stderr = proc.stderr.decode("utf-8", errors="replace").strip()
|
||||
|
||||
if proc.returncode != 0:
|
||||
raise RuntimeError(f"Gemini 调用失败 (code={proc.returncode}): {stderr or '无错误输出'}")
|
||||
|
||||
if stderr:
|
||||
# 某些 CLI 可能在 stderr 打印警告，保留但不中断
|
||||
print(f"⚠️ Gemini 警告: {stderr}", file=sys.stderr)
|
||||
|
||||
if not stdout:
|
||||
raise RuntimeError("Gemini 未返回内容")
|
||||
|
||||
# 去除多余行，只保留非空行并合并
|
||||
lines = [ln for ln in stdout.splitlines() if ln.strip()]
|
||||
return " ".join(lines).strip()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
args = parse_args()
|
||||
input_dir = args.input.resolve()
|
||||
if not input_dir.exists() or not input_dir.is_dir():
|
||||
print(f"❌ 输入目录不存在: {input_dir}")
|
||||
sys.exit(1)
|
||||
|
||||
output_path = args.output or (input_dir / "prompts.jsonl")
|
||||
output_path = output_path.resolve()
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
md_files = sorted(f for f in input_dir.iterdir() if f.suffix == ".md")
|
||||
if not md_files:
|
||||
print(f"⚠️ 未找到任何 .md 文件: {input_dir}")
|
||||
sys.exit(0)
|
||||
|
||||
results = []
|
||||
for md in md_files:
|
||||
content = md.read_text(encoding="utf-8")
|
||||
if args.verbose:
|
||||
print(f"→ 处理 {md.name}")
|
||||
try:
|
||||
json_line = run_gemini(content, args.model, args.gemini_cmd)
|
||||
results.append(json_line)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print(f"❌ 处理失败 {md.name}: {exc}", file=sys.stderr)
|
||||
|
||||
with output_path.open("w", encoding="utf-8") as f:
|
||||
for line in results:
|
||||
f.write(line + "\n")
|
||||
|
||||
print(f"✅ 完成：{len(results)} 条 → {output_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,36 @@
|
||||
import json
|
||||
import pandas as pd
|
||||
|
||||
input_file = "prompt_jsonl/prompt_docs_refactored.jsonl"
|
||||
output_file = "prompt_excel/prompt_docs_refactored.xlsx"
|
||||
|
||||
def process():
|
||||
data_by_cat = {}
|
||||
with open(input_file, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
if not line.strip(): continue
|
||||
item = json.loads(line)
|
||||
cat = item['category']
|
||||
if cat not in data_by_cat:
|
||||
data_by_cat[cat] = []
|
||||
|
||||
# Reconstruct the JSON string for the cell as it was in original Excel
|
||||
cell_data = {
|
||||
"title": item.get('title', ''),
|
||||
"content": item.get('content', '')
|
||||
}
|
||||
data_by_cat[cat].append(json.dumps(cell_data, ensure_ascii=False))
|
||||
|
||||
with pd.ExcelWriter(output_file, engine='openpyxl') as writer:
|
||||
# Sort categories to keep a consistent order
|
||||
sorted_cats = sorted(data_by_cat.keys())
|
||||
for cat in sorted_cats:
|
||||
items = data_by_cat[cat]
|
||||
# Each item in its own row, column 0
|
||||
df = pd.DataFrame(items)
|
||||
df.to_excel(writer, sheet_name=cat, index=False, header=False)
|
||||
|
||||
print(f"Excel created: {output_file}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
process()
|
||||
@@ -0,0 +1,73 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
将 prompt_docs 目录下的 md 文件转换为 JSONL 格式
|
||||
|
||||
用法:
|
||||
python md_to_jsonl.py <prompt_docs目录>
|
||||
python md_to_jsonl.py prompt_docs/prompt_docs_2025_1222_004537
|
||||
"""
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
OUTPUT_DIR = REPO_ROOT / "prompt_jsonl"
|
||||
|
||||
def convert(docs_dir: Path):
|
||||
prompts_dir = docs_dir / "prompts"
|
||||
if not prompts_dir.exists():
|
||||
print(f"❌ 找不到 prompts 目录: {prompts_dir}")
|
||||
return
|
||||
|
||||
# 输出文件名基于输入目录名
|
||||
output_file = OUTPUT_DIR / f"{docs_dir.name}.jsonl"
|
||||
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
records = []
|
||||
for category_dir in sorted(prompts_dir.iterdir()):
|
||||
if not category_dir.is_dir():
|
||||
continue
|
||||
|
||||
m = re.match(r'\((\d+)\)_(.+)', category_dir.name)
|
||||
cat_id, cat_name = (m.groups() if m else (0, category_dir.name))
|
||||
|
||||
for md_file in sorted(category_dir.glob("*.md")):
|
||||
if md_file.name == "index.md":
|
||||
continue
|
||||
|
||||
fm = re.match(r'\((\d+),(\d+)\)_(.+)\.md', md_file.name)
|
||||
if not fm:
|
||||
continue
|
||||
|
||||
row, col, title = fm.groups()
|
||||
content = md_file.read_text(encoding='utf-8')
|
||||
|
||||
records.append({
|
||||
"category_id": int(cat_id),
|
||||
"category": cat_name,
|
||||
"row": int(row),
|
||||
"col": int(col),
|
||||
"title": title[:80],
|
||||
"content": content
|
||||
})
|
||||
|
||||
with open(output_file, 'w', encoding='utf-8') as f:
|
||||
for r in records:
|
||||
f.write(json.dumps(r, ensure_ascii=False) + '\n')
|
||||
|
||||
print(f"✅ 转换完成: {len(records)} 条 → {output_file}")
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print(__doc__)
|
||||
sys.exit(1)
|
||||
|
||||
docs_dir = Path(sys.argv[1])
|
||||
if not docs_dir.is_absolute():
|
||||
docs_dir = REPO_ROOT / docs_dir
|
||||
|
||||
convert(docs_dir)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,154 @@
|
||||
import json
|
||||
import os
|
||||
|
||||
mapping = {
|
||||
# 编程技术
|
||||
"软件工程，glue_coding_用提示词": "编程技术",
|
||||
"前端复刻流程": "编程技术",
|
||||
"输入转单行JSON": "编程技术",
|
||||
"序列图生成": "编程技术",
|
||||
"流程图": "编程技术",
|
||||
"函数化万物": "编程技术",
|
||||
"编程知识库": "编程技术",
|
||||
"网页UI逆向分析提示词": "编程技术",
|
||||
"用户优化前端设计": "编程技术",
|
||||
"图像特征提取": "编程技术",
|
||||
"前端通用设计": "编程技术",
|
||||
|
||||
# 逻辑工具箱
|
||||
"哲学工具箱": "逻辑工具箱",
|
||||
"逻辑工具箱": "逻辑工具箱",
|
||||
"批判性思维分析": "逻辑工具箱",
|
||||
"思维模型": "逻辑工具箱",
|
||||
"政治批判工具箱": "逻辑工具箱",
|
||||
"未来视角": "逻辑工具箱",
|
||||
"层级结构分析": "逻辑工具箱",
|
||||
"问题分类识别": "逻辑工具箱",
|
||||
"分析": "逻辑工具箱",
|
||||
"终极本质分析": "逻辑工具箱",
|
||||
"事实核查": "逻辑工具箱",
|
||||
"关键词图谱": "逻辑工具箱",
|
||||
"语言分析元prompt": "逻辑工具箱",
|
||||
"逻辑分析": "逻辑工具箱",
|
||||
"黄金圈解释": "逻辑工具箱",
|
||||
"谋士": "逻辑工具箱",
|
||||
"经验": "逻辑工具箱",
|
||||
"道": "逻辑工具箱",
|
||||
"法": "逻辑工具箱",
|
||||
"术": "逻辑工具箱",
|
||||
"器": "逻辑工具箱",
|
||||
"心经口诀创作提示词": "逻辑工具箱",
|
||||
"临界知识": "逻辑工具箱",
|
||||
"项目分析": "逻辑工具箱",
|
||||
"对话提问": "逻辑工具箱",
|
||||
"思维导图": "逻辑工具箱",
|
||||
|
||||
# 内容创作
|
||||
"文案逆向": "内容创作",
|
||||
"x_prompt收集": "内容创作",
|
||||
"x提示词收集": "内容创作",
|
||||
"x爆款文案生成器": "内容创作",
|
||||
"推文制作提示词": "内容创作",
|
||||
"李继刚文选": "内容创作",
|
||||
"解释提示词": "内容创作",
|
||||
"一句话描述任何内容": "内容创作",
|
||||
"子弹总结": "内容创作",
|
||||
"文本转md语法电子书处理": "内容创作",
|
||||
"排版和图片，视频转文本": "内容创作",
|
||||
"艺术风格描述": "内容创作",
|
||||
"视频生成提示词": "内容创作",
|
||||
"图片逆向": "内容创作",
|
||||
"排版": "内容创作",
|
||||
"内容提炼": "内容创作",
|
||||
"简讯提示词": "内容创作",
|
||||
"艺术": "内容创作",
|
||||
"人话写作": "内容创作",
|
||||
"小红书": "内容创作",
|
||||
"组织语言": "内容创作",
|
||||
"正向人物生平报告官方文案": "内容创作",
|
||||
"gemini字幕处理": "内容创作",
|
||||
|
||||
# 学习教育
|
||||
"学习提示词": "学习教育",
|
||||
"学习用提示词": "学习教育",
|
||||
"ai学习用提示词": "学习教育",
|
||||
"书籍结构化分析": "学习教育",
|
||||
"典籍句子学习": "学习教育",
|
||||
"anki卡片格式输出": "学习教育",
|
||||
"notebookllm用提示词": "学习教育",
|
||||
"英文学习": "学习教育",
|
||||
"速成学习": "学习教育",
|
||||
"论文解读": "学习教育",
|
||||
"真传一句话": "学习教育",
|
||||
"学习音频": "学习教育",
|
||||
"豆包听书": "学习教育",
|
||||
"最小知识框架": "学习教育",
|
||||
|
||||
# 商业分析
|
||||
"grok商业金融分析提示词": "商业分析",
|
||||
"投资调研": "商业分析",
|
||||
"行业分析": "商业分析",
|
||||
"需求对齐": "商业分析",
|
||||
"需求结构化描述": "商业分析",
|
||||
"麦肯锡行业分析": "商业分析",
|
||||
"产品策略": "商业分析",
|
||||
"行业咨询": "商业分析",
|
||||
"需求解析": "商业分析",
|
||||
"SOP制作": "商业分析",
|
||||
|
||||
# 提示词工程
|
||||
"元提示词": "提示词工程",
|
||||
"提示词模块": "提示词工程",
|
||||
"根据内容逆向提示词": "提示词工程",
|
||||
"系统提示词": "提示词工程",
|
||||
"AI使用思维": "提示词工程",
|
||||
"使用ai的思维": "提示词工程",
|
||||
"最小字数系统提示词": "提示词工程",
|
||||
"ChatGPT": "提示词工程",
|
||||
"Reddit提示词": "提示词工程",
|
||||
"好prompt生成器": "提示词工程",
|
||||
"思维协议": "提示词工程",
|
||||
"grok抓取提示词": "提示词工程",
|
||||
|
||||
# 其他
|
||||
"AI_交易系统提示词": "综合杂项",
|
||||
"面向CZ": "综合杂项",
|
||||
}
|
||||
|
||||
id_map = {
|
||||
"编程技术": 1,
|
||||
"逻辑工具箱": 2,
|
||||
"内容创作": 3,
|
||||
"学习教育": 4,
|
||||
"商业分析": 5,
|
||||
"提示词工程": 6,
|
||||
"综合杂项": 7
|
||||
}
|
||||
|
||||
input_file = "prompt_jsonl/prompt_docs_2025_1222_004537.jsonl"
|
||||
output_file = "prompt_jsonl/prompt_docs_refactored.jsonl"
|
||||
|
||||
def process():
|
||||
stats = {}
|
||||
with open(input_file, 'r', encoding='utf-8') as fin, \
|
||||
open(output_file, 'w', encoding='utf-8') as fout:
|
||||
for line in fin:
|
||||
if not line.strip(): continue
|
||||
data = json.loads(line)
|
||||
old_cat = data.get('category', '')
|
||||
new_cat = mapping.get(old_cat, "综合杂项")
|
||||
|
||||
# Keep original category in tags if it doesn't exist?
|
||||
# Or just replace. The user said "只调整 'category'"
|
||||
data['category'] = new_cat
|
||||
data['category_id'] = id_map.get(new_cat, 7)
|
||||
|
||||
fout.write(json.dumps(data, ensure_ascii=False) + '\n')
|
||||
stats[new_cat] = stats.get(new_cat, 0) + 1
|
||||
|
||||
print("Refactor complete.")
|
||||
for cat, count in stats.items():
|
||||
print(f"{cat}: {count}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
process()
|
||||
@@ -0,0 +1,49 @@
|
||||
import json
|
||||
import shutil
|
||||
from collections import defaultdict
|
||||
|
||||
input_file = "prompt_jsonl/prompt_docs_refactored.jsonl"
|
||||
output_file = "prompt_jsonl/prompt_docs_refactored_reindexed.jsonl"
|
||||
backup_file = "prompt_jsonl/prompt_docs_refactored_before_reindex.jsonl.bak"
|
||||
|
||||
def reindex_rows():
|
||||
# 1. Backup
|
||||
shutil.copy(input_file, backup_file)
|
||||
print(f"Backup created: {backup_file}")
|
||||
|
||||
# 2. Load and Group
|
||||
items_by_cat = defaultdict(list)
|
||||
|
||||
with open(input_file, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
if not line.strip(): continue
|
||||
item = json.loads(line)
|
||||
cat = item.get('category', 'Uncategorized')
|
||||
items_by_cat[cat].append(item)
|
||||
|
||||
# 3. Sort and Reindex
|
||||
total_items = 0
|
||||
with open(output_file, 'w', encoding='utf-8') as f:
|
||||
# Sort categories for consistent file order
|
||||
for cat in sorted(items_by_cat.keys()):
|
||||
items = items_by_cat[cat]
|
||||
# Sort items by their OLD row to preserve relative order
|
||||
items.sort(key=lambda x: x.get('row', 0))
|
||||
|
||||
# Reassign row numbers starting from 1
|
||||
for i, item in enumerate(items):
|
||||
item['row'] = i + 1
|
||||
f.write(json.dumps(item, ensure_ascii=False) + '\n')
|
||||
total_items += 1
|
||||
|
||||
print(f"Category '{cat}': re-indexed {len(items)} items.")
|
||||
|
||||
print(f"Re-indexed file written: {output_file}")
|
||||
print(f"Total items: {total_items}")
|
||||
|
||||
# Overwrite original
|
||||
shutil.move(output_file, input_file)
|
||||
print(f"Overwritten original file: {input_file}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
reindex_rows()
|
||||
@@ -0,0 +1,11 @@
|
||||
# 提示词库管理系统依赖包
|
||||
pandas==2.1.4
|
||||
openpyxl==3.1.2
|
||||
google-auth==2.22.0
|
||||
google-auth-oauthlib==1.0.0
|
||||
google-auth-httplib2==0.1.0
|
||||
google-api-python-client==2.96.0
|
||||
PyYAML==6.0.1
|
||||
python-dotenv==1.0.0
|
||||
rich==13.7.1
|
||||
InquirerPy==0.3.4
|
||||
@@ -0,0 +1,188 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
start_convert.py
|
||||
|
||||
Launcher that orchestrates conversions between Excel workbooks and prompt documents
|
||||
using the following conventions:
|
||||
|
||||
Input locations (relative to repo root):
|
||||
- ./prompt_excel/ # place .xlsx files here for Excel → Docs
|
||||
- ./prompt_docs/ # place prompt folders here for Docs → Excel
|
||||
|
||||
Output locations (under repo root, named by source file/folder mtime):
|
||||
- ./prompt_docs_YYYYMMDD_HHMMSS/ # Excel → Docs results (copies of prompts/*)
|
||||
- ./prompt_excel_YYYYMMDD_HHMMSS/ # Docs → Excel results (rebuilt.xlsx)
|
||||
|
||||
Usage:
|
||||
# Auto mode: if there are .xlsx under prompt_excel, run Excel→Docs;
|
||||
# if there is a docs set under prompt_docs, run Docs→Excel.
|
||||
python prompt-library/scripts/start_convert.py
|
||||
|
||||
# Force a mode:
|
||||
python prompt-library/scripts/start_convert.py --mode excel2docs
|
||||
python prompt-library/scripts/start_convert.py --mode docs2excel
|
||||
|
||||
Notes:
|
||||
- No interactive prompts; behavior is driven by the file presence and CLI flags
|
||||
- Requires pandas, openpyxl, PyYAML (see scripts/requirements.txt)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import importlib.util
|
||||
import shutil
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import List
|
||||
|
||||
|
||||
def ts_from_path(p: Path) -> str:
|
||||
st = p.stat()
|
||||
# Prefer creation/birth time when available; fall back to mtime
|
||||
ts = getattr(st, "st_birthtime", None)
|
||||
if ts is None:
|
||||
# On Windows, st_ctime is creation; on Linux it's inode change time
|
||||
# We still prefer mtime for consistency if birthtime is unavailable.
|
||||
ts = st.st_mtime
|
||||
# Format: YYYY_MMDD_HHMMSS per requirement example 2025_0102_2309
|
||||
return datetime.fromtimestamp(ts).strftime("%Y_%m%d_%H%M%S")
|
||||
|
||||
|
||||
def load_module(py_path: Path, module_name: str):
|
||||
spec = importlib.util.spec_from_file_location(module_name, str(py_path))
|
||||
if spec is None or spec.loader is None:
|
||||
raise RuntimeError(f"Unable to load module: {py_path}")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[module_name] = module
|
||||
spec.loader.exec_module(module) # type: ignore
|
||||
return module
|
||||
|
||||
|
||||
def run_excel_to_docs_for_file(excel_path: Path, prompt_library_dir: Path, out_root: Path) -> Path:
|
||||
convert_path = prompt_library_dir / "scripts" / "convert_local.py"
|
||||
mod = load_module(convert_path, "convert_local")
|
||||
|
||||
project_root = prompt_library_dir.parent
|
||||
# Prepare snapshot output directory under repo_root/prompt_docs/
|
||||
base_dir = out_root / "prompt_docs"
|
||||
base_dir.mkdir(parents=True, exist_ok=True)
|
||||
out_dir = base_dir / f"prompt_docs_{ts_from_path(excel_path)}"
|
||||
if out_dir.exists():
|
||||
shutil.rmtree(out_dir)
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
converter = mod.ExcelPromptConverter(
|
||||
project_root=project_root,
|
||||
prompt_library_dir=prompt_library_dir,
|
||||
excel_path=excel_path,
|
||||
category_name="prompt-category",
|
||||
config_path=None,
|
||||
output_root=out_dir,
|
||||
)
|
||||
converter.convert()
|
||||
|
||||
return out_dir
|
||||
|
||||
|
||||
def run_docs_to_excel_for_dir(prompts_dir: Path, scripts_dir: Path, out_root: Path) -> Path:
|
||||
docs2excel_path = scripts_dir / "docs_to_excel.py"
|
||||
mod = load_module(docs2excel_path, "docs_to_excel")
|
||||
|
||||
# Determine timestamp from folder creation (or mtime fallback)
|
||||
base_dir = out_root / "prompt_excel"
|
||||
base_dir.mkdir(parents=True, exist_ok=True)
|
||||
ts_fmt = ts_from_path(prompts_dir)
|
||||
out_dir = base_dir / f"prompt_excel_{ts_fmt}"
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
out_path = out_dir / "rebuilt.xlsx"
|
||||
|
||||
# Resolve actual prompts root (support either the prompts/ subfolder or direct sheet folders)
|
||||
prompts_root = prompts_dir / "prompts" if (prompts_dir / "prompts").exists() else prompts_dir
|
||||
# Invoke module's main via argparse emulation
|
||||
sys.argv = [str(docs2excel_path), "--prompts-dir", str(prompts_root), "--out", str(out_path)]
|
||||
mod.main() # type: ignore
|
||||
|
||||
return out_dir
|
||||
|
||||
|
||||
def find_xlsx_files(input_excel_dir: Path) -> List[Path]:
|
||||
if not input_excel_dir.exists():
|
||||
return []
|
||||
return sorted([p for p in input_excel_dir.iterdir() if p.is_file() and p.suffix.lower() in {".xlsx"}], key=lambda p: p.stat().st_mtime)
|
||||
|
||||
|
||||
def has_prompt_files(input_docs_dir: Path) -> bool:
|
||||
if not input_docs_dir.exists():
|
||||
return False
|
||||
for p in input_docs_dir.rglob("*.md"):
|
||||
if p.name.startswith("(") and ")_" in p.name:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Start conversion between Excel and prompt docs")
|
||||
parser.add_argument("--mode", choices=["auto", "excel2docs", "docs2excel"], default="auto")
|
||||
parser.add_argument("--excel-dir", default="prompt_excel", help="Input directory containing .xlsx files")
|
||||
parser.add_argument("--docs-dir", default="prompt_docs", help="Input directory containing prompt folders")
|
||||
parser.add_argument("--select", type=str, default=None, help="Optional path to a specific Excel file or prompts folder to convert")
|
||||
args = parser.parse_args()
|
||||
|
||||
script_path = Path(__file__).resolve()
|
||||
prompt_library_dir = script_path.parent.parent # repo root (prompt-library)
|
||||
project_root = prompt_library_dir # use prompt-library as root for I/O
|
||||
|
||||
input_excel_dir = (prompt_library_dir / args.excel_dir).resolve()
|
||||
input_docs_dir = (prompt_library_dir / args.docs_dir).resolve()
|
||||
|
||||
ran_any = False
|
||||
|
||||
if args.mode in ("auto", "excel2docs"):
|
||||
# If user explicitly selected a file, prefer it
|
||||
if args.select:
|
||||
sel = Path(args.select)
|
||||
if not sel.is_absolute():
|
||||
sel = (project_root / sel).resolve()
|
||||
if sel.is_file() and sel.suffix.lower() == ".xlsx":
|
||||
out_dir = run_excel_to_docs_for_file(sel, prompt_library_dir, project_root)
|
||||
rel = out_dir.relative_to(prompt_library_dir)
|
||||
print(f"✅ Excel→Docs OK: {sel.name} → {rel}")
|
||||
ran_any = True
|
||||
else:
|
||||
xlsx_files = find_xlsx_files(input_excel_dir)
|
||||
for xlsx in xlsx_files:
|
||||
out_dir = run_excel_to_docs_for_file(xlsx, prompt_library_dir, project_root)
|
||||
rel = out_dir.relative_to(prompt_library_dir)
|
||||
print(f"✅ Excel→Docs OK: {xlsx.name} → {rel}")
|
||||
ran_any = True
|
||||
|
||||
if args.mode in ("auto", "docs2excel"):
|
||||
if args.select:
|
||||
sel = Path(args.select)
|
||||
if not sel.is_absolute():
|
||||
sel = (project_root / sel).resolve()
|
||||
if sel.exists() and sel.is_dir():
|
||||
out_dir = run_docs_to_excel_for_dir(sel, prompt_library_dir / "scripts", project_root)
|
||||
rel = out_dir.relative_to(prompt_library_dir)
|
||||
# show sel relative as well when possible
|
||||
try:
|
||||
sel_rel = Path(sel).relative_to(prompt_library_dir)
|
||||
except Exception:
|
||||
sel_rel = Path(sel)
|
||||
print(f"✅ Docs→Excel OK: {sel_rel} → {rel}")
|
||||
ran_any = True
|
||||
else:
|
||||
if has_prompt_files(input_docs_dir):
|
||||
out_dir = run_docs_to_excel_for_dir(input_docs_dir, prompt_library_dir / "scripts", project_root)
|
||||
rel = out_dir.relative_to(prompt_library_dir)
|
||||
print(f"✅ Docs→Excel OK: {args.docs_dir} → {rel}")
|
||||
ran_any = True
|
||||
|
||||
if not ran_any:
|
||||
print("ℹ️ Nothing to do. Place .xlsx under ./prompt_excel or prompt docs under ./prompt_docs, or use --mode to force.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,109 @@
|
||||
import json
|
||||
import os
|
||||
from collections import defaultdict
|
||||
|
||||
jsonl_path = "prompt_jsonl/prompt_docs_refactored.jsonl"
|
||||
docs_root = "prompt_docs/prompt_docs_refactored/prompts"
|
||||
|
||||
def verify():
|
||||
print("=== 开始全面完整性检查 ===\n")
|
||||
|
||||
# 1. JSONL 数据加载与基础检查
|
||||
if not os.path.exists(jsonl_path):
|
||||
print(f"❌ 错误: JSONL 文件不存在: {jsonl_path}")
|
||||
return
|
||||
|
||||
data = []
|
||||
with open(jsonl_path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
if line.strip():
|
||||
try:
|
||||
data.append(json.loads(line))
|
||||
except json.JSONDecodeError:
|
||||
print(f"❌ 错误: 发现无效的 JSON 行: {line[:50]}...")
|
||||
|
||||
total_items = len(data)
|
||||
print(f"✅ JSONL 读取成功，共 {total_items} 条数据。")
|
||||
|
||||
# 2. 规则验证
|
||||
errors = []
|
||||
categories = defaultdict(list)
|
||||
|
||||
expected_categories = {
|
||||
"内容创作", "商业分析", "学习教育", "提示词工程", "综合杂项", "编程技术", "逻辑工具箱"
|
||||
}
|
||||
|
||||
for item in data:
|
||||
cat = item.get('category')
|
||||
row = item.get('row')
|
||||
col = item.get('col')
|
||||
title = item.get('title')
|
||||
content = item.get('content')
|
||||
|
||||
# 收集分类数据用于后续分析
|
||||
categories[cat].append(row)
|
||||
|
||||
# 检查 1: 分类合法性
|
||||
if cat not in expected_categories:
|
||||
errors.append(f"❌ 未知分类: '{cat}' (Title: {title[:20]}...)")
|
||||
|
||||
# 检查 2: 列归位 (col == 1)
|
||||
if col != 1:
|
||||
errors.append(f"❌ 列未归位: Category '{cat}', Row {row}, Col {col} (应为 1)")
|
||||
|
||||
# 检查 3: 内容完整性 (简单检查)
|
||||
if not title:
|
||||
errors.append(f"⚠️ 警告: 标题为空 (Category '{cat}', Row {row})")
|
||||
if not content or len(content) < 5:
|
||||
errors.append(f"⚠️ 警告: 内容过短或为空 (Category '{cat}', Row {row}, Content len: {len(content) if content else 0})")
|
||||
|
||||
# 检查 4: 行连续性
|
||||
print("\n--- 分类与行号连续性检查 ---")
|
||||
for cat, rows in categories.items():
|
||||
rows.sort()
|
||||
count = len(rows)
|
||||
if count == 0:
|
||||
print(f"⚠️ 分类 '{cat}' 为空")
|
||||
continue
|
||||
|
||||
max_row = rows[-1]
|
||||
expected_rows = list(range(1, count + 1))
|
||||
|
||||
status = "✅ 正常"
|
||||
if rows != expected_rows:
|
||||
status = "❌ 异常 (行号不连续或重复)"
|
||||
errors.append(f"行号错误: {cat} (Expect 1-{count}, Got max {max_row})")
|
||||
|
||||
print(f"{cat.ljust(10)}: {count} 条 | Max Row: {max_row} | {status}")
|
||||
|
||||
# 3. 文件系统同步检查
|
||||
print("\n--- 文档文件同步检查 ---")
|
||||
files_found = 0
|
||||
if os.path.exists(docs_root):
|
||||
for root, dirs, files in os.walk(docs_root):
|
||||
for file in files:
|
||||
if file.endswith(".md") and not file.startswith("index"):
|
||||
files_found += 1
|
||||
else:
|
||||
print(f"❌ 文档目录不存在: {docs_root}")
|
||||
|
||||
print(f"JSONL 条目数: {total_items}")
|
||||
print(f"Markdown 文件数: {files_found}")
|
||||
|
||||
if total_items == files_found:
|
||||
print("✅ 文件数量一致")
|
||||
else:
|
||||
print(f"❌ 文件数量不匹配! (差值: {files_found - total_items})")
|
||||
errors.append("文件系统数量与 JSONL 不一致")
|
||||
|
||||
# 4. 总结
|
||||
print("\n=== 检查总结 ===")
|
||||
if not errors:
|
||||
print("🎉 完美！所有检查通过。数据结构完整、规范。")
|
||||
else:
|
||||
print(f"发现 {len(errors)} 个问题，请检视：")
|
||||
for err in errors:
|
||||
print(err)
|
||||
|
||||
if __name__ == "__main__":
|
||||
verify()
|
||||
Reference in New Issue
Block a user