A White-Box False Positive Adversarial Attack Method on Contrastive Loss Based Offline Handwritten Signature Verification Models
In this paper, we tackle the challenge of white-box false positive adversarial attacks on contrastive loss based offline handwritten signature verification models. We propose a novel attack method that treats the attack as a style transfer between closely related but distinct writing styles. To guide the generation of deceptive images, we introduce two new loss functions that enhance the attack success rate by perturbing the Euclidean distance between the embedding vectors of the original and synthesized samples, while ensuring minimal perturbations by reducing the difference between the generated image and the original image. Our method demonstrates state-of-the-art performance in white-box attacks on contrastive loss based offline handwritten signature verification models, as evidenced by our experiments. The key contributions of this paper include a novel false positive attack method, two new loss functions, effective style transfer in handwriting styles, and superior performance in white-box false positive attacks compared to other white-box attack methods.
Updated: 2024-02-09 23:47:02
标题: 基于对比损失的离线手写签名验证模型的白盒误报对抗攻击方法
摘要: 在这篇论文中，我们解决了基于对比损失的离线手写签名验证模型中白盒假阳性对抗攻击的挑战。我们提出了一种新颖的攻击方法，将攻击视为在紧密相关但不同的书写风格之间进行风格转移。为了引导欺骗性图像的生成，我们引入了两个新的损失函数，通过扰动原始样本和合成样本的嵌入向量之间的欧氏距离，提高攻击成功率，同时通过减少生成图像和原始图像之间的差异来保证最小扰动。我们的方法在基于对比损失的离线手写签名验证模型的白盒攻击中表现出最先进的性能，这在我们的实验中得到证实。本文的主要贡献包括一种新颖的假阳性攻击方法、两个新的损失函数、有效的书写风格转移以及与其他白盒攻击方法相比在白盒假阳性攻击中表现出色的性能。
更新时间: 2024-02-09 23:47:02
领域: cs.CV,cs.AI,cs.CR,cs.LG
Epsilon*: Privacy Metric for Machine Learning Models
We introduce Epsilon*, a new privacy metric for measuring the privacy risk of a single model instance prior to, during, or after deployment of privacy mitigation strategies. The metric requires only black-box access to model predictions, does not require training data re-sampling or model re-training, and can be used to measure the privacy risk of models not trained with differential privacy. Epsilon* is a function of true positive and false positive rates in a hypothesis test used by an adversary in a membership inference attack. We distinguish between quantifying the privacy loss of a trained model instance, which we refer to as empirical privacy, and quantifying the privacy loss of the training mechanism which produces this model instance. Existing approaches in the privacy auditing literature provide lower bounds for the latter, while our metric provides an empirical lower bound for the former by relying on an (${\epsilon}$, ${\delta}$)-type of quantification of the privacy of the trained model instance. We establish a relationship between these lower bounds and show how to implement Epsilon* to avoid numerical and noise amplification instability. We further show in experiments on benchmark public data sets that Epsilon* is sensitive to privacy risk mitigation by training with differential privacy (DP), where the value of Epsilon* is reduced by up to 800% compared to the Epsilon* values of non-DP trained baseline models. This metric allows privacy auditors to be independent of model owners, and enables visualizing the privacy-utility landscape to make informed decisions regarding the trade-offs between model privacy and utility.
Updated: 2024-02-09 23:32:58
标题: Epsilon*: 机器学习模型的隐私度量标准
摘要: 我们引入Epsilon*，一种新的隐私度量标准，用于衡量在隐私缓解策略部署之前、期间或之后，单个模型实例的隐私风险。该度量标准仅需要对模型预测进行黑盒访问，不需要训练数据重新取样或模型重新训练，并且可用于衡量未经差分隐私训练的模型的隐私风险。Epsilon*是对敌对方在成员推断攻击中使用的假设检验中的真阳性率和假阳性率的函数。我们区分了量化训练模型实例的隐私损失，我们称之为经验隐私，以及量化产生该模型实例的训练机制的隐私损失。隐私审计文献中的现有方法为后者提供了下界，而我们的度量标准通过依赖于训练模型实例的隐私的（ε，δ）-型量化，为前者提供了一个经验下界。我们建立了这些下界之间的关系，并展示了如何实施Epsilon*以避免数值和噪声放大不稳定性。我们进一步在基准公共数据集上的实验中展示，Epsilon*对通过差分隐私（DP）训练进行隐私风险缓解是敏感的，其中Epsilon*的值相比非DP训练基线模型的Epsilon*值降低了高达800%。该度量标准使隐私审计人员能够独立于模型所有者，并且能够可视化隐私-效用景观，以便做出关于模型隐私和效用之间权衡的知情决策。
更新时间: 2024-02-09 23:32:58
领域: cs.LG,cs.CR,cs.DS
RAMP: Boosting Adversarial Robustness Against Multiple $l_p$ Perturbations
There is considerable work on improving robustness against adversarial attacks bounded by a single $l_p$ norm using adversarial training (AT). However, the multiple-norm robustness (union accuracy) of AT models is still low. We observe that simultaneously obtaining good union and clean accuracy is hard since there are tradeoffs between robustness against multiple $l_p$ perturbations, and accuracy/robustness/efficiency. By analyzing the tradeoffs from the lens of distribution shifts, we identify the key tradeoff pair among $l_p$ attacks to boost efficiency and design a logit pairing loss to improve the union accuracy. Next, we connect natural training with AT via gradient projection, to find and incorporate useful information from natural training into AT, which moderates the accuracy/robustness tradeoff. Combining our contributions, we propose a framework called \textbf{RAMP}, to boost the robustness against multiple $l_p$ perturbations. We show \textbf{RAMP} can be easily adapted for both robust fine-tuning and full AT. For robust fine-tuning, \textbf{RAMP} obtains a union accuracy up to $53.5\%$ on CIFAR-10, and $29.7\%$ on ImageNet. For training from scratch, \textbf{RAMP} achieves SOTA union accuracy of $44.6\%$ and relatively good clean accuracy of $81.2\%$ on ResNet-18 against AutoAttack on CIFAR-10.
Updated: 2024-02-09 23:29:54
标题: RAMP：增强对多个$l_p$扰动的对抗性鲁棒性
摘要: 有大量工作致力于通过对抗性训练（AT）提高针对单个$l_p$范数界限的对抗性攻击的鲁棒性。然而，AT模型的多范数鲁棒性（联合准确性）仍然较低。我们观察到同时获得良好的联合准确性和干净准确性是困难的，因为在多个$l_p$扰动之间存在鲁棒性、准确性/鲁棒性/效率之间的权衡。通过从分布转移的角度分析权衡，我们确定了在$l_p$攻击中提高效率的关键权衡对，并设计了一种logit配对损失来提高联合准确性。接下来，我们通过梯度投影将自然训练与AT联系起来，以找到并将有用信息从自然训练中整合到AT中，从而调节准确性/鲁棒性的权衡。结合我们的贡献，我们提出了一个名为\textbf{RAMP}的框架，以提高针对多个$l_p$扰动的鲁棒性。我们展示\textbf{RAMP}可以很容易地适应鲁棒微调和完整的AT。对于鲁棒微调，\textbf{RAMP}在CIFAR-10上获得了高达$53.5\%$的联合准确性，在ImageNet上为$29.7\%$。对于从头开始训练，\textbf{RAMP}在ResNet-18上对抗AutoAttack在CIFAR-10上获得了$44.6\%$的SOTA联合准确性和相对较好的干净准确性为$81.2\%$。
更新时间: 2024-02-09 23:29:54
领域: cs.LG,cs.CR
Towards Principled Assessment of Tabular Data Synthesis Algorithms
Data synthesis has been advocated as an important approach for utilizing data while protecting data privacy. A large number of tabular data synthesis algorithms (which we call synthesizers) have been proposed. Some synthesizers satisfy Differential Privacy, while others aim to provide privacy in a heuristic fashion. A comprehensive understanding of the strengths and weaknesses of these synthesizers remains elusive due to lacking principled evaluation metrics and missing head-to-head comparisons of newly developed synthesizers that take advantage of diffusion models and large language models with state-of-the-art marginal-based synthesizers. In this paper, we present a principled and systematic evaluation framework for assessing tabular data synthesis algorithms. Specifically, we examine and critique existing evaluation metrics, and introduce a set of new metrics in terms of fidelity, privacy, and utility to address their limitations. Based on the proposed metrics, we also devise a unified objective for tuning, which can consistently improve the quality of synthetic data for all methods. We conducted extensive evaluations of 8 different types of synthesizers on 12 datasets and identified some interesting findings, which offer new directions for privacy-preserving data synthesis.
Updated: 2024-02-09 22:07:59
标题: 朝向基于原则的表格数据合成算法评估
摘要: 数据合成被提倡为一种重要的方法，可以利用数据同时保护数据隐私。已经提出了大量的表格数据合成算法（我们称之为合成器）。一些合成器满足差分隐私，而其他一些则旨在以一种启发式方式提供隐私保护。由于缺乏有原则的评估指标和缺少新开发的利用扩散模型和大型语言模型的合成器与最先进的基于边缘的合成器进行直接比较，对这些合成器的优势和劣势的全面理解仍然难以捉摸。 在本文中，我们提出了一个有原则和系统的评估框架，用于评估表格数据合成算法。具体来说，我们审查和批评现有的评估指标，并引入一组新的指标，以解决它们的局限性，分别是保真度、隐私性和效用。基于提出的指标，我们还设计了一个统一的调整目标，可以一致地提高所有方法的合成数据质量。我们在12个数据集上对8种不同类型的合成器进行了广泛评估，并发现了一些有趣的发现，为保护隐私的数据合成提供了新的方向。
更新时间: 2024-02-09 22:07:59
领域: cs.CR,cs.DB,cs.LG
CiFlow: Dataflow Analysis and Optimization of Key Switching for Homomorphic Encryption
Homomorphic encryption (HE) is a privacy-preserving computation technique that enables computation on encrypted data. Today, the potential of HE remains largely unrealized as it is impractically slow, preventing it from being used in real applications. A major computational bottleneck in HE is the key-switching operation, accounting for approximately 70% of the overall HE execution time and involving a large amount of data for inputs, intermediates, and keys. Prior research has focused on hardware accelerators to improve HE performance, typically featuring large on-chip SRAMs and high off-chip bandwidth to deal with large scale data. In this paper, we present a novel approach to improve key-switching performance by rigorously analyzing its dataflow. Our primary goal is to optimize data reuse with limited on-chip memory to minimize off-chip data movement. We introduce three distinct dataflows: Max-Parallel (MP), Digit-Centric (DC), and Output-Centric (OC), each with unique scheduling approaches for key-switching computations. Through our analysis, we show how our proposed Output-Centric technique can effectively reuse data by significantly lowering the intermediate key-switching working set and alleviating the need for massive off-chip bandwidth. We thoroughly evaluate the three dataflows using the RPU, a recently published vector processor tailored for ring processing algorithms, which includes HE. This evaluation considers sweeps of bandwidth and computational throughput, and whether keys are buffered on-chip or streamed. With OC, we demonstrate up to 4.16x speedup over the MP dataflow and show how OC can save 16x on-chip SRAM by streaming keys for minimal performance penalty.
Updated: 2024-02-09 21:37:13
标题: CiFlow：同态加密关键切换的数据流分析和优化
摘要: 同态加密(HE)是一种保护隐私的计算技术，可以在加密数据上进行计算。今天，HE的潜力仍然大部分未被实现，因为它在实际应用中速度太慢，无法使用。HE中的一个主要计算瓶颈是密钥切换操作，约占整体HE执行时间的70%，涉及大量输入、中间数据和密钥。先前的研究集中在硬件加速器上，以提高HE性能，通常具有大型片上静态随机访问存储器(SRAM)和高片外带宽，以处理大规模数据。在本文中，我们提出了一种通过严格分析其数据流来改善密钥切换性能的新方法。我们的主要目标是通过有限的片上内存优化数据重用，以最小化片外数据移动。我们引入了三种不同的数据流：最大并行(MP)、数字中心(DC)和输出中心(OC)，每种都具有针对密钥切换计算的独特调度方法。通过我们的分析，我们展示了我们提出的输出中心技术如何通过显著降低中间密钥切换工作集来有效地重用数据，并减轻对大量片外带宽的需求。我们使用RPU对这三种数据流进行了全面评估，RPU是一种专为环形处理算法（包括HE）定制的最近发布的向量处理器。这种评估考虑了带宽和计算吞吐量的扫描，以及密钥是在片上缓冲还是流式传输。通过OC，我们展示了比MP数据流高达4.16倍的加速，并展示了如何通过流式传输密钥来节省16倍的片上SRAM，以最小化性能损失。
更新时间: 2024-02-09 21:37:13
领域: cs.CR,cs.AR,cs.PF
On Modular Algorithms and Butterfly Operations in Number Theoretic Transform
Number theoretic transform (NTT) has been a very useful tool in computations for number theory, algebra and cryptography. Its performance affects some post-quantum cryptosystems. In this paper, we discuss the butterfly operation of NTT. This basic module of NTT requires heavy modular arithmetics. Montgomery reduction is commonly used in this setting. Recently several variants of Montgomery algorithm have been proposed for the purpose of speeding up NTT. We observe that the Chinese remainder theorem (CRT) can be involved in this type of algorithms in natural and transparent ways. In the first part of the paper, a framework of using CRT to model Montgomery type algorithms is described. The derivation of these algorithms as well as their correctness are all treated in the CRT framework. Under our approach, some problems of a modular reduction algorithm (published in IACR Transactions on Cryptographic Hardware and Embedded Systems, doi:10.46586/tches.v2022.i4.614-636 ) are identified, and a counterexample is generated to show that the algorithm is incorrect. In the second part of the paper, we modify a modular multiplication algorithm of Plantard to suite the butterfly structure by Scott, an improved computation of the butterfly module for NTT is obtained. Experiments show that the method performs better compared to NTT implementations using previous popular methods.
Updated: 2024-02-09 20:12:01
标题: 关于模算法和蝴蝶操作在数论变换中的应用
摘要: 数论变换（NTT）在数论、代数和密码学的计算中是非常有用的工具。它的性能影响着一些后量子密码系统。本文讨论了NTT的蝴蝶操作。NTT的这个基本模块需要大量的模运算。蒙哥马利约简在这种情况下通常被使用。最近为加速NTT提出了几种蒙哥马利算法的变体。我们观察到中国余数定理（CRT）可以自然而透明地参与到这种类型的算法中。在论文的第一部分中，描述了使用CRT来建模蒙哥马利类型算法的框架。这些算法的推导以及它们的正确性都在CRT框架中处理。在我们的方法下，鉴别出了一个模归约算法的一些问题（发表在IACR密码硬件和嵌入系统交易中，doi:10.46586/tches.v2022.i4.614-636），并生成了一个反例来展示该算法是不正确的。在论文的第二部分中，我们修改了Plantard的模乘法算法以适应Scott的蝴蝶结构，从而获得了一种改进的NTT蝴蝶模块的计算方法。实验表明，与使用先前流行方法的NTT实现相比，该方法表现更好。
更新时间: 2024-02-09 20:12:01
领域: cs.CR
FedMLSecurity: A Benchmark for Attacks and Defenses in Federated Learning and Federated LLMs
This paper introduces FedSecurity, an end-to-end benchmark designed to simulate adversarial attacks and corresponding defense mechanisms in Federated Learning (FL). FedSecurity comprises two pivotal components: FedAttacker, which facilitates the simulation of a variety of attacks during FL training, and FedDefender, which implements defensive mechanisms to counteract these attacks. As an open-source library, FedSecurity enhances its usability compared to from-scratch implementations that focus on specific attack/defense scenarios based on the following features: i) It offers extensive customization options to accommodate a broad range of machine learning models (e.g., Logistic Regression, ResNet, and GAN) and FL optimizers (e.g., FedAVG, FedOPT, and FedNOVA); ii) it enables exploring the variability in the effectiveness of attacks and defenses across different datasets and models; and iii) it supports flexible configuration and customization through a configuration file and some provided APIs. We further demonstrate FedSecurity's utility and adaptability through federated training of Large Language Models (LLMs), showcasing its potential to impact a wide range of complex applications.
Updated: 2024-02-09 19:57:05
标题: FedMLSecurity：联邦学习和联邦LLM中攻击和防御的基准测试
摘要: 本文介绍了FedSecurity，这是一个端到端基准测试，旨在模拟联邦学习（FL）中的对抗攻击和相应的防御机制。FedSecurity包括两个关键组件：FedAttacker，用于在FL训练期间模拟各种攻击；和FedDefender，用于实施防御机制以抵消这些攻击。作为一个开源库，与专注于特定攻击/防御情景的从头开始实现相比，FedSecurity通过以下特点增强了其可用性：i）它提供了广泛的定制选项，以适应各种机器学习模型（例如逻辑回归、ResNet和GAN）和FL优化器（例如FedAVG、FedOPT和FedNOVA）；ii）它使得能够探索在不同数据集和模型之间攻击和防御效果的可变性；iii）它通过配置文件和一些提供的API支持灵活的配置和定制。我们进一步通过对大语言模型（LLMs）进行联邦训练展示了FedSecurity的实用性和适应性，展示了它对广泛复杂应用的潜力影响。
更新时间: 2024-02-09 19:57:05
领域: cs.CR,cs.AI
RQP-SGD: Differential Private Machine Learning through Noisy SGD and Randomized Quantization
The rise of IoT devices has prompted the demand for deploying machine learning at-the-edge with real-time, efficient, and secure data processing. In this context, implementing machine learning (ML) models with real-valued weight parameters can prove to be impractical particularly for large models, and there is a need to train models with quantized discrete weights. At the same time, these low-dimensional models also need to preserve privacy of the underlying dataset. In this work, we present RQP-SGD, a new approach for privacy-preserving quantization to train machine learning models for low-memory ML-at-the-edge. This approach combines differentially private stochastic gradient descent (DP-SGD) with randomized quantization, providing a measurable privacy guarantee in machine learning. In particular, we study the utility convergence of implementing RQP-SGD on ML tasks with convex objectives and quantization constraints and demonstrate its efficacy over deterministic quantization. Through experiments conducted on two datasets, we show the practical effectiveness of RQP-SGD.
Updated: 2024-02-09 18:34:08
标题: RQP-SGD：通过SGD噪声和随机量化实现差分私密机器学习
摘要: 物联网设备的兴起促使了对在边缘部署机器学习进行实时、高效和安全数据处理的需求。在这种情况下，用具有实值权重参数的机器学习（ML）模型可能会变得不切实际，特别是对于大型模型，有必要使用量化的离散权重来训练模型。同时，这些低维模型还需要保护底层数据集的隐私。在这项工作中，我们提出了一种新方法RQP-SGD，用于隐私保护量化，以训练用于低内存ML-at-the-edge的机器学习模型。该方法结合了差分私有随机梯度下降（DP-SGD）和随机量化，为机器学习提供了可衡量的隐私保证。特别是，我们研究了在具有凸目标和量化约束的ML任务上实施RQP-SGD的效用收敛，并证明了其在确定性量化方面的有效性。通过对两个数据集进行的实验，我们展示了RQP-SGD的实际有效性。
更新时间: 2024-02-09 18:34:08
领域: cs.LG,cs.AI,cs.CR
Toward More Generalized Malicious URL Detection Models
This paper reveals a data bias issue that can severely affect the performance while conducting a machine learning model for malicious URL detection. We describe how such bias can be identified using interpretable machine learning techniques, and further argue that such biases naturally exist in the real world security data for training a classification model. We then propose a debiased training strategy that can be applied to most deep-learning based models to alleviate the negative effects from the biased features. The solution is based on the technique of self-supervised adversarial training to train deep neural networks learning invariant embedding from biased data. We conduct a wide range of experiments to demonstrate that the proposed strategy can lead to significantly better generalization capability for both CNN-based and RNN-based detection models.
Updated: 2024-02-09 17:20:19
标题: 朝着更通用的恶意URL检测模型前进
摘要: 这篇论文揭示了一个数据偏差问题，可能严重影响进行恶意URL检测的机器学习模型的性能。我们描述了如何利用可解释的机器学习技术来识别这种偏差，并进一步论证了在真实世界的安全数据中训练分类模型时这种偏差自然存在。然后，我们提出了一种去偏差的训练策略，可以应用于大多数基于深度学习的模型，以减轻来自偏差特征的负面影响。解决方案基于自监督对抗训练技术，用于从偏差数据中训练深度神经网络学习不变嵌入。我们进行了广泛的实验，以证明所提出的策略可以显著提高基于CNN和RNN的检测模型的泛化能力。
更新时间: 2024-02-09 17:20:19
领域: cs.LG,cs.CR
Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks
AI-based code generators have become pivotal in assisting developers in writing software starting from natural language (NL). However, they are trained on large amounts of data, often collected from unsanitized online sources (e.g., GitHub, HuggingFace). As a consequence, AI models become an easy target for data poisoning, i.e., an attack that injects malicious samples into the training data to generate vulnerable code. To address this threat, this work investigates the security of AI code generators by devising a targeted data poisoning strategy. We poison the training data by injecting increasing amounts of code containing security vulnerabilities and assess the attack's success on different state-of-the-art models for code generation. Our study shows that AI code generators are vulnerable to even a small amount of poison. Notably, the attack success strongly depends on the model architecture and poisoning rate, whereas it is not influenced by the type of vulnerabilities. Moreover, since the attack does not impact the correctness of code generated by pre-trained models, it is hard to detect. Lastly, our work offers practical insights into understanding and potentially mitigating this threat.
Updated: 2024-02-09 16:28:40
标题: AI代码生成器中的漏洞：探索针对性数据中毒攻击
摘要: 基于人工智能的代码生成器已经成为协助开发人员从自然语言（NL）开始编写软件的关键工具。然而，它们通常是在大量数据的基础上进行训练的，这些数据经常来自未经处理的在线来源（如GitHub、HuggingFace）。因此，人工智能模型成为数据毒化的易受攻击目标，即注入恶意样本到训练数据中以生成易受攻击的代码。 为了应对这一威胁，本研究通过设计有针对性的数据毒化策略来调查人工智能代码生成器的安全性。我们通过注入包含安全漏洞的代码来毒化训练数据，并评估攻击对不同最先进的代码生成模型的成功率。我们的研究表明，即使是少量毒素，人工智能代码生成器也容易受到攻击。值得注意的是，攻击的成功程度强烈依赖于模型架构和毒化率，而不受漏洞类型的影响。此外，由于攻击不会影响预训练模型生成的代码的正确性，因此很难检测。最后，我们的工作为了解和潜在减轻这一威胁提供了实用见解。
更新时间: 2024-02-09 16:28:40
领域: cs.CR,cs.AI
HoneyDOC: An Efficient Honeypot Architecture Enabling All-Round Design
Honeypots are designed to trap the attacker with the purpose of investigating its malicious behavior. Owing to the increasing variety and sophistication of cyber attacks, how to capture high-quality attack data has become a challenge in the context of honeypot area. All-round honeypots, which mean significant improvement in sensibility, countermeasure and stealth, are necessary to tackle the problem. In this paper, we propose a novel honeypot architecture termed HoneyDOC to support all-round honeypot design and implementation. Our HoneyDOC architecture clearly identifies three essential independent and collaborative modules, Decoy, Captor and Orchestrator. Based on the efficient architecture, a Software-Defined Networking (SDN) enabled honeypot system is designed, which supplies high programmability for technically sustaining the features for capturing high-quality data. A proof-of-concept system is implemented to validate its feasibility and effectiveness. The experimental results show the benefits by using the proposed architecture comparing to the previous honeypot solutions.
Updated: 2024-02-09 16:27:45
标题: HoneyDOC：一种能够实现全方位设计的高效蜜罐架构
摘要: 蜜罐旨在诱捕攻击者，以调查其恶意行为。由于网络攻击的种类和复杂性不断增加，如何捕获高质量的攻击数据已成为蜜罐领域的挑战。全面的蜜罐，意味着在敏感性、对策和隐蔽性方面有显著改进，是解决问题的必要条件。本文提出了一种新颖的蜜罐架构，称为HoneyDOC，以支持全面的蜜罐设计和实施。我们的HoneyDOC架构清晰地确定了三个关键的独立和协作模块，即诱饵、捕捉器和编排器。基于高效的架构，设计了一种支持软件定义网络（SDN）的蜜罐系统，为捕获高质量数据提供了高程度的可编程性。我们实施了一个概念验证系统来验证其可行性和有效性。实验结果表明，使用所提出的架构比以往的蜜罐解决方案具有更多的优势。
更新时间: 2024-02-09 16:27:45
领域: cs.CR,cs.NI,C.2.0; C.2.1
On Differentially Private Subspace Estimation Without Distributional Assumptions
Private data analysis faces a significant challenge known as the curse of dimensionality, leading to increased costs. However, many datasets possess an inherent low-dimensional structure. For instance, during optimization via gradient descent, the gradients frequently reside near a low-dimensional subspace. If the low-dimensional structure could be privately identified using a small amount of points, we could avoid paying (in terms of privacy and accuracy) for the high ambient dimension. On the negative side, Dwork, Talwar, Thakurta, and Zhang (STOC 2014) proved that privately estimating subspaces, in general, requires an amount of points that depends on the dimension. But Singhal and Steinke (NeurIPS 2021) bypassed this limitation by considering points that are i.i.d. samples from a Gaussian distribution whose covariance matrix has a certain eigenvalue gap. Yet, it was still left unclear whether we could provide similar upper bounds without distributional assumptions and whether we could prove lower bounds that depend on similar eigenvalue gaps. In this work, we make progress in both directions. We formulate the problem of private subspace estimation under two different types of singular value gaps of the input data and prove new upper and lower bounds for both types. In particular, our results determine what type of gap is sufficient and necessary for estimating a subspace with an amount of points that is independent of the dimension.
Updated: 2024-02-09 15:17:53
标题: 关于无需分布假设的差分隐私子空间估计
摘要: 私人数据分析面临一个被称为维度灾难的重要挑战，导致成本增加。然而，许多数据集具有固有的低维结构。例如，在通过梯度下降进行优化时，梯度经常位于一个低维子空间附近。如果可以使用少量点私下识别低维结构，我们就可以避免在高环境维度方面支付（以隐私和准确性为代价）。 然而，Dwork、Talwar、Thakurta和Zhang（STOC 2014）证明，一般情况下，私下估计子空间需要依赖于维度的点数。但是，Singhal和Steinke（NeurIPS 2021）通过考虑来自具有一定特定特征值间隙的高斯分布的独立同分布样本的点，绕过了这一限制。然而，仍然不清楚我们是否能够在没有分布假设的情况下提供类似的上界，并且是否能够证明依赖于类似特征值间隙的下界。 在这项工作中，我们在两个不同类型的输入数据的奇异值间隙下制定私有子空间估计问题，并为两种类型证明了新的上界和下界。特别是，我们的结果确定了什么类型的间隙足够且必要以估计一个与维度无关的点数的子空间。
更新时间: 2024-02-09 15:17:53
领域: cs.LG,cs.CR,cs.DS
Maximizing NFT Incentives: References Make You Rich
In this paper, we study how to optimize existing Non-Fungible Token (NFT) incentives. Upon exploring a large number of NFT-related standards and real-world projects, we come across an unexpected finding. That is, the current NFT incentive mechanisms, often organized in an isolated and one-time-use fashion, tend to overlook their potential for scalable organizational structures. We propose, analyze, and implement a novel reference incentive model, which is inherently structured as a Directed Acyclic Graph (DAG)-based NFT network. This model aims to maximize connections (or references) between NFTs, enabling each isolated NFT to expand its network and accumulate rewards derived from subsequent or subscribed ones. We conduct both theoretical and practical analyses of the model, demonstrating its optimal utility.
Updated: 2024-02-09 15:04:16
标题: 最大化NFT激励：参考文献让你致富
摘要: 在这篇论文中，我们研究了如何优化现有的非同质化代币（NFT）激励机制。在探索大量与NFT相关的标准和现实世界项目时，我们发现了一个意外的发现。也就是说，当前NFT激励机制往往以孤立和一次性使用的方式组织，往往忽视了它们在可扩展组织结构中的潜力。 我们提出、分析并实施了一种新颖的参考激励模型，其本质上是基于有向无环图（DAG）的NFT网络结构。该模型旨在最大化NFT之间的连接（或引用），使每个孤立的NFT能够扩展其网络并累积来自后续或订阅NFT的奖励。我们对该模型进行了理论和实践分析，展示了其最佳效用。
更新时间: 2024-02-09 15:04:16
领域: cs.GT,cs.CE,cs.CR,cs.CY,econ.GN,q-fin.EC
A Method for Decrypting Data Infected with Rhysida Ransomware
Ransomware is malicious software that is a prominent global cybersecurity threat. Typically, ransomware encrypts data on a system, rendering the victim unable to decrypt it without the attacker's private key. Subsequently, victims often pay a substantial ransom to recover their data, yet some may still incur damage or loss. This study examines Rhysida ransomware, which caused significant damage in the second half of 2023, and proposes a decryption method. Rhysida ransomware employed a secure random number generator to generate the encryption key and subsequently encrypt the data. However, an implementation vulnerability existed that enabled us to regenerate the internal state of the random number generator at the time of infection. We successfully decrypted the data using the regenerated random number generator. To the best of our knowledge, this is the first successful decryption of Rhysida ransomware. We aspire for our work to contribute to mitigating the damage inflicted by the Rhysida ransomware.
Updated: 2024-02-09 14:34:09
标题: 一种解密Rhysida勒索软件感染数据的方法
摘要: 勒索软件是一种突出的全球网络安全威胁。通常，勒索软件会对系统上的数据进行加密，使受害者无法在没有攻击者私钥的情况下解密。随后，受害者通常会支付大笔赎金以恢复其数据，然而有些可能仍会遭受损失。本研究考察了在2023年下半年造成重大损害的Rhysida勒索软件，并提出了一种解密方法。Rhysida勒索软件采用了安全的随机数生成器来生成加密密钥，然后对数据进行加密。然而，在实现中存在一个漏洞，使我们能够重新生成感染时随机数生成器的内部状态。我们成功地使用重新生成的随机数生成器解密了数据。据我们所知，这是对Rhysida勒索软件的首次成功解密。我们希望我们的工作能有助于减轻由Rhysida勒索软件造成的损害。
更新时间: 2024-02-09 14:34:09
领域: cs.CR
LLM in the Shell: Generative Honeypots
Honeypots are essential tools in cybersecurity. However, most of them (even the high-interaction ones) lack the required realism to engage and fool human attackers. This limitation makes them easily discernible, hindering their effectiveness. This work introduces a novel method to create dynamic and realistic software honeypots based on Large Language Models. Preliminary results indicate that LLMs can create credible and dynamic honeypots capable of addressing important limitations of previous honeypots, such as deterministic responses, lack of adaptability, etc. We evaluated the realism of each command by conducting an experiment with human attackers who needed to say if the answer from the honeypot was fake or not. Our proposed honeypot, called shelLM, reached an accuracy of 0.92. The source code and prompts necessary for replicating the experiments have been made publicly available.
Updated: 2024-02-09 14:03:08
标题: 壳中的LLM：生成式蜜罐
摘要: 蜜罐在网络安全中是必不可少的工具。然而，大多数蜜罐（甚至包括高交互型蜜罐）缺乏必要的真实性来吸引并愚弄人类攻击者。这种限制使它们容易被识别，影响了它们的有效性。本研究介绍了一种基于大型语言模型创建动态和逼真软件蜜罐的新方法。初步结果表明，大型语言模型可以创建可信且动态的蜜罐，能够解决以前蜜罐的重要限制，如确定性响应、缺乏适应性等。我们通过对人类攻击者进行实验来评估每个命令的逼真程度，他们需要判断蜜罐的回答是否是假的。我们提出的蜜罐称为shelLM，达到了0.92的准确率。用于复制实验的源代码和提示已经公开可用。
更新时间: 2024-02-09 14:03:08
领域: cs.CR,cs.AI,cs.CL
Trust the Process: Zero-Knowledge Machine Learning to Enhance Trust in Generative AI Interactions
Generative AI, exemplified by models like transformers, has opened up new possibilities in various domains but also raised concerns about fairness, transparency and reliability, especially in fields like medicine and law. This paper emphasizes the urgency of ensuring fairness and quality in these domains through generative AI. It explores using cryptographic techniques, particularly Zero-Knowledge Proofs (ZKPs), to address concerns regarding performance fairness and accuracy while protecting model privacy. Applying ZKPs to Machine Learning models, known as ZKML (Zero-Knowledge Machine Learning), enables independent validation of AI-generated content without revealing sensitive model information, promoting transparency and trust. ZKML enhances AI fairness by providing cryptographic audit trails for model predictions and ensuring uniform performance across users. We introduce snarkGPT, a practical ZKML implementation for transformers, to empower users to verify output accuracy and quality while preserving model privacy. We present a series of empirical results studying snarkGPT's scalability and performance to assess the feasibility and challenges of adopting a ZKML-powered approach to capture quality and performance fairness problems in generative AI models.
Updated: 2024-02-09 14:00:16
标题: 相信这个过程：零知识机器学习增强对生成式人工智能交互的信任
摘要: 生成式人工智能（Generative AI），例如transformers模型，已经在各个领域开辟了新的可能性，但也引发了在公平性、透明度和可靠性方面的担忧，特别是在医学和法律等领域。本文强调了通过生成式人工智能确保这些领域的公平性和质量的紧迫性。它探讨了使用密码学技术，特别是零知识证明（ZKPs），来解决有关性能公平性和准确性的担忧，同时保护模型隐私。将ZKPs应用于机器学习模型，称为ZKML（Zero-Knowledge Machine Learning），可以独立验证由人工智能生成的内容，而不泄露敏感的模型信息，促进透明度和信任。ZKML通过为模型预测提供密码学审计跟踪，确保在用户之间实现统一性能，从而增强了人工智能的公平性。我们介绍了snarkGPT，一个用于transformers的实用ZKML实现，使用户可以验证输出的准确性和质量，同时保护模型隐私。我们提供了一系列实证结果，研究了snarkGPT的可扩展性和性能，以评估采用ZKML方法捕捉生成式人工智能模型中质量和性能公平性问题的可行性和挑战。
更新时间: 2024-02-09 14:00:16
领域: cs.LG,cs.CR
StruQ: Defending Against Prompt Injection with Structured Queries
Recent advances in Large Language Models (LLMs) enable exciting LLM-integrated applications, which perform text-based tasks by utilizing their advanced language understanding capabilities. However, as LLMs have improved, so have the attacks against them. Prompt injection attacks are an important threat: they trick the model to deviate from the original application's instructions and instead follow user directives. These attacks rely on the LLM's ability to follow instructions and inability to separate the prompts and user data. We introduce structured queries, a general approach to tackle this problem. Structured queries separate prompts and data into two channels. We implement a system that supports structured queries. This system is made of (1) a secure front-end that formats a prompt and user data into a special format, and (2) a specially trained LLM that can produce high-quality outputs from these inputs. The LLM is trained using a novel fine-tuning strategy: we convert a base (non-instruction-tuned) LLM to a structured instruction-tuned model that will only follow instructions in the prompt portion of a query. To do so, we augment standard instruction tuning datasets with examples that also include instructions in the data portion of the query, and fine-tune the model to ignore these. Our system significantly improves resistance to prompt injection attacks, with little or no impact on utility. Our code is released at https://github.com/Sizhe-Chen/PromptInjectionDefense.
Updated: 2024-02-09 12:15:51
标题: StruQ: 使用结构化查询防御提示注入
摘要: 最近大型语言模型（LLM）的进展使得集成LLM的应用程序变得更加令人兴奋，这些应用程序通过利用其先进的语言理解能力来执行基于文本的任务。然而，随着LLM的改进，针对它们的攻击也在增加。提示注入攻击是一种重要威胁：它们迷惑模型偏离原始应用程序的指令，而是遵循用户的指令。这些攻击依赖于LLM遵循指令的能力以及无法区分提示和用户数据的能力。我们引入了结构化查询，这是一种解决这个问题的通用方法。结构化查询将提示和数据分为两个通道。我们实现了一个支持结构化查询的系统。该系统由（1）一个安全的前端组成，可以将提示和用户数据格式化为特殊格式，以及（2）一个经过特殊训练的LLM，可以从这些输入中产生高质量的输出。LLM使用一种新颖的微调策略进行训练：我们将基础（非指令调整）LLM转换为结构化指令调整模型，该模型只会遵循查询的提示部分中的指令。为此，我们将标准指令调整数据集补充为还包括查询数据部分中的指令的示例，并微调模型以忽略这些指令。我们的系统显著提高了对提示注入攻击的抵抗力，对实用性几乎没有或没有影响。我们的代码发布在https://github.com/Sizhe-Chen/PromptInjectionDefense。
更新时间: 2024-02-09 12:15:51
领域: cs.CR
The SpongeNet Attack: Sponge Weight Poisoning of Deep Neural Networks
Sponge attacks aim to increase the energy consumption and computation time of neural networks deployed on hardware accelerators. Existing sponge attacks can be performed during inference via sponge examples or during training via Sponge Poisoning. Sponge examples leverage perturbations added to the model's input to increase energy and latency, while Sponge Poisoning alters the objective function of a model to induce inference-time energy/latency effects. In this work, we propose a novel sponge attack called SpongeNet. SpongeNet is the first sponge attack that is performed directly on the parameters of a pre-trained model. Our experiments show that SpongeNet can successfully increase the energy consumption of vision models with fewer samples required than Sponge Poisoning. Our experiments indicate that poisoning defenses are ineffective if not adjusted specifically for the defense against Sponge Poisoning (i.e., they decrease batch normalization bias values). Our work shows that SpongeNet is more effective on StarGAN than the state-of-the-art. Additionally, SpongeNet is stealthier than the previous Sponge Poisoning attack as it does not require significant changes in the victim model's weights. Our experiments indicate that the SpongeNet attack can be performed even when an attacker has access to only 1% of the entire dataset and reach up to 11% energy increase.
Updated: 2024-02-09 12:07:06
标题: “SpongeNet攻击：深度神经网络的海绵权重毒化”
摘要: 海绵攻击旨在增加部署在硬件加速器上的神经网络的能耗和计算时间。现有的海绵攻击可以通过海绵示例在推断期间执行，也可以通过海绵中毒在训练期间执行。海绵示例利用添加到模型输入的扰动来增加能量和延迟，而海绵中毒则改变模型的目标函数以诱导推断时的能量/延迟效果。 在这项工作中，我们提出了一种名为SpongeNet的新型海绵攻击。SpongeNet是第一个直接在预训练模型的参数上执行的海绵攻击。我们的实验表明，SpongeNet可以成功地增加视觉模型的能耗，而所需样本数量比海绵中毒更少。我们的实验表明，如果毒害防御没有专门针对对抗海绵中毒进行调整（即它们会降低批归一化偏差值），则毒害防御是无效的。我们的工作表明，SpongeNet在StarGAN上比最先进的方法更有效。此外，SpongeNet比之前的海绵中毒攻击更隐蔽，因为它不需要在受害模型的权重中进行重大更改。我们的实验表明，即使攻击者只能访问整个数据集的1%，SpongeNet攻击也可以执行，并且能量增加可以达到11%。
更新时间: 2024-02-09 12:07:06
领域: cs.CR,cs.LG
Blockchain Bribing Attacks and the Efficacy of Counterincentives
We analyze bribing attacks in distributed ledgers from a game theoretic perspective. In bribing attacks, an adversary offers to maintainers a financial reward, in exchange for instructing them on how to behave, with the goal of attacking the protocol's properties. We consider two types of bribing, depending on how the bribes are awarded: i) guided bribing, where the bribe is given as long as the bribed party behaves as instructed; ii) effective bribing, where bribes are conditional on the attack's success, w.r.t. well-defined metrics. We analyze each type of attack in a game theoretic setting and identify relevant equilibria. In guided bribing, we show that the protocol is not an equilibrium and then describe good equilibria, where the attack is unsuccessful, and a negative one, where all parties are bribed such that the attack succeeds. In effective bribing, we show that both the protocol and the "all bribed" setting are equilibria. Using the identified equilibria, we then compute bounds on the Prices of Stability and Anarchy. Our results indicate that additional mitigations are needed for guided bribing, so our analysis concludes with incentive-based mitigation techniques, namely slashing and dilution. Here, we present two positive results, that both render the protocol an equilibrium and achieve maximal welfare for all parties, and a negative result, wherein an attack becomes more plausible if it severely affects the ledger's token's market price.
Updated: 2024-02-09 11:57:38
标题: 区块链贿赂攻击及对抗激励措施的有效性
摘要: 我们从博弈论的角度分析了分布式账本中的贿赂攻击。在贿赂攻击中，对手提供给维护者一笔财务奖励，以交换指导他们如何行事的信息，从而攻击协议的特性。我们考虑了两种类型的贿赂，取决于贿赂是如何奖励的：i）引导性贿赂，其中只要被贿赂的一方按照指示行事，就会给予贿赂；ii）有效性贿赂，其中贿赂取决于攻击的成功与否，相对于明确定义的指标。我们在博弈论框架下分析了每种类型的攻击，并确定了相关的均衡。在引导性贿赂中，我们表明协议不是一个均衡，然后描述了良好的均衡，其中攻击不成功，以及一个负面的均衡，其中所有参与方都被贿赂以使攻击成功。在有效性贿赂中，我们表明协议和“所有被贿赂”设置都是均衡。利用确定的均衡，我们然后计算了稳定价格和混乱价格的上下界。我们的结果表明，引导性贿赂需要额外的缓解措施，因此我们的分析以基于激励的缓解技术，即减持和稀释，结束。在这里，我们提出了两个积极的结果，两者都使协议成为均衡，并为所有参与方实现最大利益，以及一个负面的结果，在这种情况下，如果对账本代币的市场价格产生严重影响，攻击变得更加可信。
更新时间: 2024-02-09 11:57:38
领域: cs.GT,cs.CR
Evaluating Membership Inference Attacks and Defenses in Federated Learning
Membership Inference Attacks (MIAs) pose a growing threat to privacy preservation in federated learning. The semi-honest attacker, e.g., the server, may determine whether a particular sample belongs to a target client according to the observed model information. This paper conducts an evaluation of existing MIAs and corresponding defense strategies. Our evaluation on MIAs reveals two important findings about the trend of MIAs. Firstly, combining model information from multiple communication rounds (Multi-temporal) enhances the overall effectiveness of MIAs compared to utilizing model information from a single epoch. Secondly, incorporating models from non-target clients (Multi-spatial) significantly improves the effectiveness of MIAs, particularly when the clients' data is homogeneous. This highlights the importance of considering the temporal and spatial model information in MIAs. Next, we assess the effectiveness via privacy-utility tradeoff for two type defense mechanisms against MIAs: Gradient Perturbation and Data Replacement. Our results demonstrate that Data Replacement mechanisms achieve a more optimal balance between preserving privacy and maintaining model utility. Therefore, we recommend the adoption of Data Replacement methods as a defense strategy against MIAs. Our code is available in https://github.com/Liar-Mask/FedMIA.
Updated: 2024-02-09 09:58:35
标题: 评估联邦学习中的成员推断攻击和防御
摘要: 成员推理攻击（MIAs）对联邦学习中的隐私保护构成越来越严重的威胁。半诚实的攻击者，例如服务器，可以根据观察到的模型信息确定特定样本是否属于目标客户端。本文对现有的MIAs和相应的防御策略进行了评估。我们对MIAs的评估揭示了关于MIAs趋势的两个重要发现。首先，将来自多次通信轮回的模型信息（多时态）结合起来，相比于利用单个时代的模型信息，增强了MIAs的整体有效性。其次，将来自非目标客户端的模型（多空间）整合进来显著提高了MIAs的效果，特别是当客户端的数据是同质的时候。这突显了在MIAs中考虑时空模型信息的重要性。接下来，我们通过隐私 - 效用权衡评估了两种抵御MIAs的防御机制：梯度扰动和数据替换。我们的结果表明，数据替换机制在保护隐私和维持模型效用之间实现了更优化的平衡。因此，我们建议采用数据替换方法作为抵御MIAs的防御策略。我们的代码可以在https://github.com/Liar-Mask/FedMIA找到。
更新时间: 2024-02-09 09:58:35
领域: cs.LG,cs.CR
Blockchain-based Rental Documentation Management with Audit Support
Document management in the rental market is a critical process to ensure the accuracy of financial transactions and regulatory compliance in the sector. In Portugal, the challenges include the complexity of legislation, particularly GDPR non-compliance, lack of transparency, and bureaucratic process inefficiency. With this in mind, a solution based on Hyperledger Fabric, a blockchain platform, is presented for the implementation of a document management system for the rental process. This system oversees the rental process, which consists of three phases: the application for a property by the prospective tenant through the upload of necessary documents, acceptance/rejection by the landlord of various received applications, and the creation of a report by the system, which only the auditor can request and view. The system smart contract records metadata associated with the documents (hash, owner) and coordinates requests for file access by landlords to prospective tenants. Thus, the system is responsible for creating immutable and traceable records of the entire process. The underlying platform serves as the foundation for conducting future audits. After the landlord verifies the files and accepts the rental proposal, any authorised auditor can request a report for a property by accessing the records through the final report, which includes all events that occurred during the process.
Updated: 2024-02-09 09:57:49
标题: 基于区块链的租赁文件管理与审计支持
摘要: 在租赁市场中，文件管理是一个至关重要的过程，以确保金融交易的准确性和遵守行业规定。在葡萄牙，挑战包括立法的复杂性，特别是GDPR的不遵守，缺乏透明度，以及官僚主义流程的低效性。基于此，提出了一种基于区块链平台Hyperledger Fabric的解决方案，用于实施租赁流程的文件管理系统。该系统监督租赁流程，包括三个阶段：潜在租户通过上传必要文件申请房产、房东接受/拒绝不同的申请以及系统生成报告，只有审计员可以请求和查看。系统的智能合约记录与文件相关的元数据（哈希值、所有者）并协调房东对潜在租户的文件访问请求。因此，该系统负责创建整个流程的不可变和可追溯记录。基础平台为进行未来审计提供了基础。房东验证文件并接受租赁提议后，任何授权的审计员都可以通过最终报告访问记录，请求特定房产的报告，其中包括整个流程中发生的所有事件。
更新时间: 2024-02-09 09:57:49
领域: cs.CR,cs.CY
Studious Bob Fight Back Against Jailbreaking via Prompt Adversarial Tuning
Although Large Language Models (LLMs) have achieved tremendous success in various applications, they are also susceptible to certain prompts that can induce them to bypass built-in safety measures and provide dangerous or illegal content, a phenomenon known as jailbreak. To protect LLMs from producing harmful information, various defense strategies are proposed, with most focusing on content filtering or adversarial training of models. In this paper, we propose an approach named Prompt Adversarial Tuning (PAT) to train a defense control mechanism, which is then embedded as a prefix to user prompts to implement our defense strategy. We design a training process similar to adversarial training to achieve our optimized goal, alternating between updating attack and defense controls. To our knowledge, we are the first to implement defense from the perspective of prompt tuning. Once employed, our method will hardly impact the operational efficiency of LLMs. Experiments show that our method is effective in both black-box and white-box settings, reducing the success rate of advanced attacks to nearly 0 while maintaining the benign answer rate of 80% to simple benign questions. Our work might potentially chart a new perspective for future explorations in LLM security.
Updated: 2024-02-09 09:09:39
标题: 勤奋的鲍勃通过即时对抗调整反击越狱
摘要: 尽管大型语言模型（LLMs）在各种应用中取得了巨大成功，但它们也容易受到特定提示的影响，这些提示可能导致它们绕过内置的安全措施，并提供危险或非法内容，这种现象被称为越狱。为了保护LLMs不产生有害信息，提出了各种防御策略，其中大多数集中在内容过滤或模型的对抗训练上。在本文中，我们提出了一种名为Prompt Adversarial Tuning（PAT）的方法来训练一个防御控制机制，然后将其嵌入为用户提示的前缀以实施我们的防御策略。我们设计了一个类似对抗训练的训练过程，以实现我们的优化目标，交替更新攻击和防御控制。据我们所知，我们是第一个从提示调整的角度实施防御的。一旦采用，我们的方法几乎不会影响LLMs的运行效率。实验证明我们的方法在黑盒和白盒设置中都是有效的，将高级攻击的成功率降低到接近0，同时保持对简单良性问题的良性回答率为80%。我们的工作可能为LLM安全领域的未来探索开辟了新的视角。
更新时间: 2024-02-09 09:09:39
领域: cs.LG,cs.AI,cs.CL,cs.CR
AdvART: Adversarial Art for Camouflaged Object Detection Attacks
Physical adversarial attacks pose a significant practical threat as it deceives deep learning systems operating in the real world by producing prominent and maliciously designed physical perturbations. Emphasizing the evaluation of naturalness is crucial in such attacks, as humans can readily detect and eliminate unnatural manipulations. To overcome this limitation, recent work has proposed leveraging generative adversarial networks (GANs) to generate naturalistic patches, which may not catch human's attention. However, these approaches suffer from a limited latent space which leads to an inevitable trade-off between naturalness and attack efficiency. In this paper, we propose a novel approach to generate naturalistic and inconspicuous adversarial patches. Specifically, we redefine the optimization problem by introducing an additional loss term to the cost function. This term works as a semantic constraint to ensure that the generated camouflage pattern holds semantic meaning rather than arbitrary patterns. The additional term leverages similarity metrics to construct a similarity loss that we optimize within the global objective function. Our technique is based on directly manipulating the pixel values in the patch, which gives higher flexibility and larger space compared to the GAN-based techniques that are based on indirectly optimizing the patch by modifying the latent vector. Our attack achieves superior success rate of up to 91.19\% and 72\%, respectively, in the digital world and when deployed in smart cameras at the edge compared to the GAN-based technique.
Updated: 2024-02-09 08:57:35
标题: AdvART：用于伪装物体检测攻击的对抗性艺术
摘要: 物理对抗攻击对深度学习系统构成了重大实际威胁，通过产生突出且恶意设计的物理扰动来欺骗在现实世界中运行的深度学习系统。在这种攻击中，强调自然性的评估至关重要，因为人类可以轻松检测和消除不自然的操纵。为了克服这一限制，最近的研究提出利用生成对抗网络（GANs）生成自然主义的补丁，这些补丁可能不会引起人类的注意。然而，这些方法受限于有限的潜在空间，导致自然性和攻击效率之间不可避免的权衡。在本文中，我们提出了一种新颖的方法来生成自然且不显眼的对抗性补丁。具体来说，我们通过引入额外的损失项重新定义了优化问题的成本函数。这个项作为语义约束，确保生成的伪装图案具有语义意义，而不是任意图案。额外的项利用相似性度量来构建一个相似性损失，我们在全局目标函数中优化这个损失。我们的技术基于直接操纵补丁中的像素值，与基于GAN的技术相比具有更高的灵活性和更大的空间，后者是基于间接优化补丁通过修改潜在向量。与基于GAN的技术相比，在数字世界和部署在边缘智能摄像头时，我们的攻击成功率分别高达91.19\%和72\%，取得了优越的成绩。
更新时间: 2024-02-09 08:57:35
领域: cs.CV,cs.CR
Anomaly Unveiled: Securing Image Classification against Adversarial Patch Attacks
Adversarial patch attacks pose a significant threat to the practical deployment of deep learning systems. However, existing research primarily focuses on image pre-processing defenses, which often result in reduced classification accuracy for clean images and fail to effectively counter physically feasible attacks. In this paper, we investigate the behavior of adversarial patches as anomalies within the distribution of image information and leverage this insight to develop a robust defense strategy. Our proposed defense mechanism utilizes a clustering-based technique called DBSCAN to isolate anomalous image segments, which is carried out by a three-stage pipeline consisting of Segmenting, Isolating, and Blocking phases to identify and mitigate adversarial noise. Upon identifying adversarial components, we neutralize them by replacing them with the mean pixel value, surpassing alternative replacement options. Our model-agnostic defense mechanism is evaluated across multiple models and datasets, demonstrating its effectiveness in countering various adversarial patch attacks in image classification tasks. Our proposed approach significantly improves accuracy, increasing from 38.8\% without the defense to 67.1\% with the defense against LaVAN and GoogleAp attacks, surpassing prominent state-of-the-art methods such as LGS (53.86\%) and Jujutsu (60\%)
Updated: 2024-02-09 08:52:47
标题: 揭示异常：保护图像分类免受对抗性贴片攻击
摘要: Adversarial patch attacks对于深度学习系统的实际部署构成了重大威胁。然而，现有的研究主要集中在图像预处理防御方面，这往往会导致干净图像的分类准确度降低，并且未能有效地对抗物理可行的攻击。在本文中，我们研究了对抗性贴片作为图像信息分布中的异常的行为，并利用这一洞见开发了一种强大的防御策略。我们提出的防御机制利用一种基于聚类的技术称为DBSCAN来隔离异常图像片段，这是通过一个包括分割、隔离和阻止阶段的三阶段流程来实现的，以识别和减轻对抗性噪声。在识别对抗性组件后，我们通过将其替换为平均像素值来中和它们，超越了替代替换选项。我们的模型无关的防御机制在多个模型和数据集上进行了评估，证明了其在对抗性贴片攻击中的有效性，可以提高图像分类任务的准确度。我们提出的方法显著提高了准确度，从没有防御时的38.8\%提高到使用防御时的67.1\%，对抗LaVAN和GoogleAp攻击，超过了突出的最新方法，如LGS（53.86\%）和Jujutsu（60\%）。
更新时间: 2024-02-09 08:52:47
领域: cs.CV,cs.CR
Privacy Profiles for Private Selection
Private selection mechanisms (e.g., Report Noisy Max, Sparse Vector) are fundamental primitives of differentially private (DP) data analysis with wide applications to private query release, voting, and hyperparameter tuning. Recent work (Liu and Talwar, 2019; Papernot and Steinke, 2022) has made significant progress in both generalizing private selection mechanisms and tightening their privacy analysis using modern numerical privacy accounting tools, e.g., R\'enyi DP. But R\'enyi DP is known to be lossy when $(\epsilon,\delta)$-DP is ultimately needed, and there is a trend to close the gap by directly handling privacy profiles, i.e., $\delta$ as a function of $\epsilon$ or its equivalent dual form known as $f$-DPs. In this paper, we work out an easy-to-use recipe that bounds the privacy profiles of ReportNoisyMax and PrivateTuning using the privacy profiles of the base algorithms they corral. Numerically, our approach improves over the RDP-based accounting in all regimes of interest and leads to substantial benefits in end-to-end private learning experiments. Our analysis also suggests new distributions, e.g., binomial distribution for randomizing the number of rounds that leads to more substantial improvements in certain regimes.
Updated: 2024-02-09 08:31:46
标题: 隐私配置文件用于私人选择
摘要: 私人选择机制（例如，Report Noisy Max，Sparse Vector）是差分私密（DP）数据分析的基本原语，具有广泛的应用，包括私人查询发布、投票和超参数调整。最近的工作（Liu和Talwar，2019；Papernot和Steinke，2022）在泛化私人选择机制和使用现代数值隐私会计工具（例如，Rényi DP）紧密化其隐私分析方面取得了显著进展。但是已知Rényi DP在最终需要（ε，δ）-DP时是有损的，并且有一种趋势通过直接处理隐私配置文件来缩小差距，即将δ作为ε的函数或其等效的双重形式，即f-DPs。在本文中，我们提出了一个易于使用的方法，通过限制它们所包含的基本算法的隐私配置文件，来限制Report Noisy Max和PrivateTuning的隐私配置文件。在数值上，我们的方法在所有感兴趣的范围内都优于基于RDP的会计方法，并在端到端私人学习实验中带来了实质性的好处。我们的分析还指出了新的分布，例如，用于随机化回合数量的二项分布，在某些范围内可以带来更大的改进。
更新时间: 2024-02-09 08:31:46
领域: cs.CR,cs.LG
Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakage
The advancement of large language models (LLMs) brings notable improvements across various applications, while simultaneously raising concerns about potential private data exposure. One notable capability of LLMs is their ability to form associations between different pieces of information, but this raises concerns when it comes to personally identifiable information (PII). This paper delves into the association capabilities of language models, aiming to uncover the factors that influence their proficiency in associating information. Our study reveals that as models scale up, their capacity to associate entities/information intensifies, particularly when target pairs demonstrate shorter co-occurrence distances or higher co-occurrence frequencies. However, there is a distinct performance gap when associating commonsense knowledge versus PII, with the latter showing lower accuracy. Despite the proportion of accurately predicted PII being relatively small, LLMs still demonstrate the capability to predict specific instances of email addresses and phone numbers when provided with appropriate prompts. These findings underscore the potential risk to PII confidentiality posed by the evolving capabilities of LLMs, especially as they continue to expand in scale and power.
Updated: 2024-02-09 05:31:11
标题: 量化大型语言模型的关联能力及其对隐私泄露的影响
摘要: 大型语言模型（LLMs）的进步在各种应用中带来了显著的改进，同时也引起了对潜在私人数据曝露的担忧。LLM的一个显著能力是它们能够在不同信息之间建立关联，但是当涉及个人可识别信息（PII）时，这引发了担忧。本文探讨了语言模型的关联能力，旨在揭示影响它们在关联信息方面的熟练程度的因素。我们的研究表明，随着模型规模的扩大，它们关联实体/信息的能力会加强，特别是当目标对表现出较短的共现距离或更高的共现频率时。然而，在关联常识知识与PII方面存在明显的性能差距，后者的准确性较低。尽管准确预测PII的比例相对较小，但LLM仍然表现出在提供适当提示时可以预测特定实例的电子邮件地址和电话号码的能力。这些发现强调了随着LLM不断扩大规模和能力，对PII保密性的潜在风险。
更新时间: 2024-02-09 05:31:11
领域: cs.CL,cs.AI,cs.CR
High Epsilon Synthetic Data Vulnerabilities in MST and PrivBayes
Synthetic data generation (SDG) has become increasingly popular as a privacy-enhancing technology. It aims to maintain important statistical properties of its underlying training data, while excluding any personally identifiable information. There have been a whole host of SDG algorithms developed in recent years to improve and balance both of these aims. Many of these algorithms provide robust differential privacy guarantees. However, we show here that if the differential privacy parameter $\varepsilon$ is set too high, then unambiguous privacy leakage can result. We show this by conducting a novel membership inference attack (MIA) on two state-of-the-art differentially private SDG algorithms: MST and PrivBayes. Our work suggests that there are vulnerabilities in these generators not previously seen, and that future work to strengthen their privacy is advisable. We present the heuristic for our MIA here. It assumes knowledge of auxiliary "population" data, and also assumes knowledge of which SDG algorithm was used. We use this information to adapt the recent DOMIAS MIA uniquely to MST and PrivBayes. Our approach went on to win the SNAKE challenge in November 2023.
Updated: 2024-02-09 05:13:37
标题: MST和PrivBayes中高ε合成数据的漏洞
摘要: 合成数据生成（SDG）作为一种增强隐私保护的技术，已经变得越来越受欢迎。它旨在保持其基础训练数据的重要统计特性，同时排除任何个人可识别信息。近年来已经开发了许多SDG算法，旨在改进并平衡这两个目标。许多这些算法提供了强大的差分隐私保证。 然而，我们在这里展示，如果差分隐私参数ε设置过高，可能会导致明确的隐私泄露。我们通过对两种最先进的差分私密SDG算法：MST和PrivBayes进行新颖的成员推断攻击（MIA）来展示这一点。我们的工作表明，这些生成器存在以前未见的漏洞，并建议未来的工作加强其隐私保护。 我们在这里介绍我们MIA的启发式方法。它假设具有辅助“人口”数据的知识，并且还假设知道使用了哪个SDG算法。我们利用这些信息将最近的DOMIAS MIA独特地适应于MST和PrivBayes。我们的方法在2023年11月赢得了SNAKE挑战。
更新时间: 2024-02-09 05:13:37
领域: cs.CR
Passwords Are Meant to Be Secret: A Practical Secure Password Entry Channel for Web Browsers
Password-based authentication faces various security and usability issues. Password managers help alleviate some of these issues by enabling users to manage their passwords effectively. However, malicious client-side scripts and browser extensions can steal passwords after they have been autofilled by the manager into the web page. In this paper, we explore what role the password manager can take in preventing the theft of autofilled credentials without requiring a change to user behavior. To this end, we identify a threat model for password exfiltration and then use this threat model to explore the design space for secure password entry implemented using a password manager. We identify five potential designs that address this issue, each with varying security and deployability tradeoffs. Our analysis shows the design that best balances security and usability is for the manager to autofill a fake password and then rely on the browser to replace the fake password with the actual password immediately before the web request is handed over to the operating system to be transmitted over the network. This removes the ability for malicious client-side scripts or browser extensions to access and exfiltrate the real password. We implement our design in the Firefox browser and conduct experiments, which show that it successfully thwarts malicious scripts and extensions on 97\% of the Alexa top 1000 websites, while also maintaining the capability to revert to default behavior on the remaining websites, avoiding functionality regressions. Most importantly, this design is transparent to users, requiring no change to user behavior.
Updated: 2024-02-09 03:21:14
标题: 密码应该是秘密的：一种实用的安全密码输入通道，用于网络浏览器
摘要: 基于密码的身份验证面临着各种安全性和可用性问题。密码管理器通过使用户能够有效管理他们的密码，有助于缓解其中一些问题。然而，恶意的客户端脚本和浏览器扩展可以在密码经由密码管理器自动填充到网页后窃取密码。本文探讨了密码管理器在防止自动填充凭据被盗取方面所起的作用，而无需改变用户行为。为此，我们确定了一个密码外泄的威胁模型，然后利用这个威胁模型来探索使用密码管理器实现安全密码输入的设计空间。我们确定了五种潜在的设计方案来解决这个问题，每种方案都有不同的安全性和可部署性权衡。我们的分析显示，最好平衡安全性和可用性的设计是让管理器自动填充一个虚假密码，然后依靠浏览器在网络请求交给操作系统传输之前立即用真实密码替换虚假密码。这消除了恶意的客户端脚本或浏览器扩展访问和外泄真实密码的能力。我们在Firefox浏览器中实现了我们的设计，并进行了实验，结果表明它成功地挫败了97%的Alexa前1000个网站上的恶意脚本和扩展，同时也保持了在其余网站上恢复到默认行为的能力，避免了功能退化。最重要的是，这种设计对用户是透明的，无需改变用户行为。
更新时间: 2024-02-09 03:21:14
领域: cs.CR
Barycentric and Pairwise Renyi Quantum Leakage
Barycentric and pairwise quantum Renyi leakages are proposed as two measures of information leakage for privacy and security analysis in quantum computing and communication systems. These quantities both require minimal assumptions on the eavesdropper, i.e., they do not make any assumptions on the eavesdropper's attack strategy or the statistical prior on the secret or private classical data encoded in the quantum system. They also satisfy important properties of positivity, independence, post-processing inequality, and unitary invariance. The barycentric quantum Renyi leakage can be computed by solving a semi-definite program and the pairwise quantum Renyi leakage possesses an explicit formula. The barycentric and pairwise quantum Renyi leakages form upper bounds on the maximal quantum leakage, the sandwiched quantum $\alpha$-mutual information, the accessible information, and the Holevo's information. Furthermore, differentially-private quantum channels are shown to bound these measures of information leakage. Global and local depolarizing channels, that are common models of noise in quantum computing and communication, restrict private or secure information leakage. Finally, a privacy-utility trade-off formula in quantum machine learning using variational circuits is developed. The privacy guarantees can only be strengthened, i.e., information leakage can only be reduced, if the performance degradation grows larger and vice versa.
Updated: 2024-02-09 03:09:33
标题: 质心和成对的Renyi量子泄漏
摘要: 文献提出了重心量子Renyi泄漏和成对量子Renyi泄漏作为量子计算和通信系统中隐私和安全分析的信息泄漏的两种度量。这些量都对窃听者的假设要求最小，即它们不对窃听者的攻击策略或在量子系统中编码的秘密或私有经典数据的统计先验做出任何假设。它们还满足积极性、独立性、后处理不等式和酉不变性等重要属性。重心量子Renyi泄漏可以通过解半定规划来计算，而成对量子Renyi泄漏具有显式公式。重心和成对量子Renyi泄漏形成了最大量子泄漏、夹在量子α-互信息、可访问信息和Holevo信息的上界。此外，显示了差分私密量子通道限制这些信息泄漏度量。全局和本地去极化通道，这是量子计算和通信中噪声的常见模型，限制了私密或安全信息泄漏。最后，发展了使用变分电路的量子机器学习中的隐私-效用权衡公式。只有在性能降级增大时，隐私保证才能被加强，即信息泄漏才能被减少，反之亦然。
更新时间: 2024-02-09 03:09:33
领域: quant-ph,cs.CR,cs.IT,math.IT
Deep Learning Based Face Recognition Method using Siamese Network
Achieving state-of-the-art results in face verification systems typically hinges on the availability of labeled face training data, a resource that often proves challenging to acquire in substantial quantities. In this research endeavor, we proposed employing Siamese networks for face recognition, eliminating the need for labeled face images. We achieve this by strategically leveraging negative samples alongside nearest neighbor counterparts, thereby establishing positive and negative pairs through an unsupervised methodology. The architectural framework adopts a VGG encoder, trained as a double branch siamese network. Our primary aim is to circumvent the necessity for labeled face image data, thus proposing the generation of training pairs in an entirely unsupervised manner. Positive training data are selected within a dataset based on their highest cosine similarity scores with a designated anchor, while negative training data are culled in a parallel fashion, though drawn from an alternate dataset. During training, the proposed siamese network conducts binary classification via cross-entropy loss. Subsequently, during the testing phase, we directly extract face verification scores from the network's output layer. Experimental results reveal that the proposed unsupervised system delivers a performance on par with a similar but fully supervised baseline.
Updated: 2024-02-09 02:32:57
标题: 使用孪生网络的基于深度学习的人脸识别方法
摘要: 在面部验证系统中实现最先进的结果通常取决于标记的面部训练数据的可用性，这是一种往往难以大量获取的资源。在这项研究工作中，我们提出利用连体网络进行面部识别，从而消除了对标记的面部图像的需求。我们通过在最近邻对旁边策略性地利用负样本，从而通过无监督方法建立正样本和负样本对来实现这一目标。该架构框架采用VGG编码器，作为一个双分支连体网络进行训练。我们的主要目标是规避对标记的面部图像数据的必要性，因此提出完全无监督的方式生成训练对。在数据集中，根据它们与指定锚点的最高余弦相似度分数选择正训练数据，而负训练数据以类似方式被剔除，尽管来自另一个数据集。在训练过程中，提出的连体网络通过交叉熵损失进行二元分类。随后，在测试阶段，我们直接从网络的输出层提取面部验证分数。实验结果表明，提出的无监督系统提供了与类似但完全监督基准相当的性能。
更新时间: 2024-02-09 02:32:57
领域: cs.CV,cs.AI,cs.CR