I'm building a parser using nom and I'd like to use nom combinators to process tokenized input.
The stream of tokens is represented by Vec<Token<S>>
, with Token
being:
#[derive(Debug, Clone, PartialEq)]
pub enum Token<S> {
Identifier(S),
String(S),
}
I tried to implement InputIter
for a vector of arbitrary values:
impl<'a, T> InputIter for Vec<T> {
type Item = T;
type Iter = Enumerate<Self::IterElem>;
type IterElem = Iter<'a, Self::Item>;
fn iter_indices(&self) -> Self::Iter {
self.iter().enumerate()
}
fn iter_elements(&self) -> Self::IterElem {
self.iter()
}
}
When I encountered the following error:
error[E0207]: the lifetime parameter `'a` is not constrained by the impl trait, self type, or predicates
--> src/lex/tokens.rs:37:6
|
37 | impl<'a, T> InputIter for Vec<T> {
| ^^ unconstrained lifetime parameter
I then tried to constrain the lifetime parameter using the type parameter Token<S>
:
impl<'a> InputIter for Vec<Token<&'a str>> {
type Item = Token<&'a str>;
type Iter = Enumerate<Self::IterElem>;
type IterElem = Iter<'a, Self::Item>;
fn iter_indices(&self) -> Self::Iter {
self.iter().enumerate()
}
fn iter_elements(&self) -> Self::IterElem {
self.iter()
}
}
Which led to the following:
error[E0117]: only traits defined in the current crate can be implemented for arbitrary types
--> src/lex/tokens.rs:51:1
|
51 | impl<'a> InputIter for Vec<Token<&'a str>> {
| ^^^^^^^^^^^^^^^^^^^^^^^-------------------
| | |
| | `std::vec::Vec` is not defined in the current crate
| impl doesn't use only types from inside the current crate
|
= note: define and implement a trait or new type instead
The error seems to imply that I can't implement an outside trait for an outside type, so I'm trying to wrap Vec<Token<S>>
into a custom struct:
struct Tokens<S> {
vec: Vec<Token<S>>,
}
impl<'a> InputIter for Tokens<&'a str> {
type Item = Token<&'a str>;
type Iter = Enumerate<Self::IterElem>;
type IterElem = Iter<'a, Self::Item>;
fn iter_indices(&self) -> Self::Iter {
self.vec.iter().enumerate()
}
fn iter_elements(&self) -> Self::IterElem {
self.vec.iter()
}
}
Which results in the following error:
error[E0271]: type mismatch resolving `<std::iter::Enumerate<std::slice::Iter<'_, lex::tokens::Token<&str>>> as std::iter::Iterator>::Item == (usize, lex::tokens::Token<&str>)`
--> src/lex/tokens.rs:56:3
|
55 | impl<'a> InputIter for Tokens<&'a str> {
| -------------------------------------- in this `impl` item
56 | type Item = Token<&'a str>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected reference, found enum `lex::tokens::Token`
|
= note: expected tuple `(_, &lex::tokens::Token<&str>)`
found tuple `(_, lex::tokens::Token<&str>)`
Does this mean I'm restricted to using a vector of references as opposed to a vector containing tokens directly? What gives? There doesn't seem to be anything in the InputIter
declaration to imply this constraint.
Is there another way to accomplish what I'm trying to do and represent token stream in a way that is compatible with nom combinators?
A reproducible example playground:
use std::iter::Enumerate;
use std::slice::Iter;
#[derive(Debug, Clone, PartialEq)]
pub enum Token<S> {
Identifier(S),
String(S),
}
struct Tokens<S> {
vec: Vec<Token<S>>,
}
pub trait InputIter {
type Item;
type Iter: Iterator<Item = (usize, Self::Item)>;
type IterElem: Iterator<Item = Self::Item>;
fn iter_indices(&self) -> Self::Iter;
fn iter_elements(&self) -> Self::IterElem;
}
impl<'a> InputIter for Tokens<&'a str> {
type Item = Token<&'a str>;
type Iter = Enumerate<Self::IterElem>;
type IterElem = Iter<'a, Token<&'a str>>;
fn iter_indices(&self) -> Self::Iter {
self.vec.iter().enumerate()
}
fn iter_elements(&self) -> Self::IterElem {
self.vec.iter()
}
}
fn main() {}
error[E0271]: type mismatch resolving `<std::slice::Iter<'_, Token<&str>> as std::iter::Iterator>::Item == Token<&str>`
--> src/main.rs:24:5
|
23 | impl<'a> InputIter for Tokens<&'a str> {
| -------------------------------------- in this `impl` item
24 | type Item = Token<&'a str>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected reference, found enum `Token`
|
= note: expected reference `&Token<&str>`
found enum `Token<&str>`
error[E0271]: type mismatch resolving `<std::iter::Enumerate<std::slice::Iter<'_, Token<&str>>> as std::iter::Iterator>::Item == (usize, Token<&str>)`
--> src/main.rs:24:5
|
23 | impl<'a> InputIter for Tokens<&'a str> {
| -------------------------------------- in this `impl` item
24 | type Item = Token<&'a str>;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected reference, found enum `Token`
|
= note: expected tuple `(_, &Token<&str>)`
found tuple `(_, Token<&str>)`