DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims and Rejections
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Claims 1, 3-10, 12, and 13 were amended in response to the Non-Final Office Action dated December 3, 2025. Claim 11 was cancelled. Claim 14 is newly added.
Claims 1, 3-10, and 12-14 are currently pending.
In light of the claim amendments, the Section 112(b) rejection of claims 11 and 12 have been withdrawn.
Claim Objections
Claims 12 and 13 are objected to because of the following informalities:
Due to the multiple uses of “plane” in the claims, claim 12 should be amended to recite: “acquiring by the electronic data processing device of the computer off-plane data that describes a position of at least one feature offset from the plane of the image, with respect to….”
Claim 12 recites “by an [or the] electronic data processing device of the computer” for several steps. These clauses should be separated by commas to make the claim clearer. For this and other reasons, claim 12 should be amended to recite:
acquiring, by an electronic data processing device of the computer, first feature-set data that describes a position of a first set of features of the tracking marker within a plane of an image obtained via an optical camera;
determining, by the electronic data processing device of the computer, grid data based on the first feature-set data, wherein the grid data describes a position of a grid pattern of the tracking marker within the plane of the image;
acquiring, by the electronic data processing device of the computer, second feature-set data that describes a position of a second set of features of the tracking marker within the plane of the image;
determining, by the electronic data processing device of the computer, identification data based on the first feature-set data and the second feature-set data, wherein the identification data describes an identity of the tracking marker;
acquiring, by the electronic data processing device of the computer, off-plane data that describes a position of at least one feature offset from the plane, with respect to the first set of features and/or with respect to the second set of features; and
determining, by the electronic data processing device of the computer, tracking data based on at least one of the first feature-set data, the second feature-set data and the off-plane data, wherein the tracking data describes a spatial position of the identified tracking marker.
Claim 13 should be amended to recite: “further comprising using, by the electronic data processing device of the computer, a monochrome mono-camera[[,]] that is configured to optically detect and distinguish the features from a front face of the tracking marker.”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 3-10, and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites that the plurality of features comprise “a second set of features being aligned with the grid pattern and defining an optical appearance….” Claim 12 later specifies that “the optical appearance [is] defined by one or more of: sizes of the plurality of features disposed at associated nodes of the grid pattern,….” This is unclear because the term “second set of features” is narrower than the “plurality of features,” which could include the first set of features in addition to the second set of features.
Examiner is interpreting claim 1 as follows: “…a second set of features being aligned with the grid pattern, the first and second sets of features defining an optical appearance of the tracking marker; and…”
Claim 14 recites similar limitations and is rejected for the reasons discussed above. Claims 3-10 depend directly or indirectly from claim 1 and are therefore rejected based on their dependency.
RESPONSE TO APPLICANT’S ARGUMENTS
The rejection of claims 11 and 12 under 35 U.S.C. 112(b) have been withdrawn in light of the claim amendments. However, a new rejection under 35 U.S.C. 112(b) of claims 1 and 3-10 is now entered as described above.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite or similarly recite:
[a] determining grid data based on the first feature-set data, wherein the grid data describes a position of a grid pattern of the tracking marker within the plane of the image;
[b] determining identification data based on the first feature-set data and the second feature-set data, wherein the identification data describes an identity of the tracking marker;
[c] determining tracking data based on at least one of the first feature-set data, the second feature-set data and the off-plane data, wherein the tracking data describes a spatial position of the identified tracking marker.
Claim limitation [a], as drafted and under their broadest reasonable interpretation, recite a mathematical concept. (MPEP 2106.04(a)(2)(I) (see, e.g., Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (although the claims did not recite a particular mathematical formula, the court held “[w]ithout additional limitations, a process that employs mathematical algorithms to manipulate existing information to generate additional information is not patent eligible.”)). For example, determining grid data based on the first feature-set data involves identifying locations of the features using geometric calculations.
Claim limitations [b], as drafted and under their broadest reasonable interpretation, recite a mathematical concept and/or a mental process. (MPEP 2106.04(a)(2)(I) (see, e.g., Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (although the claims did not recite a particular mathematical formula, the court held “[w]ithout additional limitations, a process that employs mathematical algorithms to manipulate existing information to generate additional information is not patent eligible.”)). For example, determining identification data based on the first feature-set data and the second feature-set data, wherein the identification data describes an identity of the tracking marker includes identifying the information that is associated with that configuration of features.
Claim limitations [c], as drafted and under their broadest reasonable interpretation, recite a mathematical concept and/or a mental process. (MPEP 2106.04(a)(2)(I) (see, e.g., Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (although the claims did not recite a particular mathematical formula, the court held “[w]ithout additional limitations, a process that employs mathematical algorithms to manipulate existing information to generate additional information is not patent eligible.”)). For example, determining tracking data based on at least one of the first feature-set data, the second feature-set data and the off-plane data, wherein the tracking data describes a spatial position of the identified tracking marker requires performing various 3D calculations to determine the position of the marker.
The next question is to consider whether the claims integrate the judicial exception into a practical application. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception. (MPEP 2106.04(d)). In this case, some additional elements/steps to consider include acquiring first feature-set data that describes a position of a first set of features of the tracking marker within a plane of an image obtained via an optical camera; acquiring second features-set data that describes a position of a second set of features of the tracking marker within the plane of the image, particularly with respect to the first set of features; and acquiring off-plane data that describes a position of at least one feature offset from the plane, with respect to the first set of features and/or with respect to the second set of features. Another additional element to consider is that the various steps are performed by an electronic data processing device of the computer.
Here, the judicial exception is not integrated into a practical application. These claim limitations recite insignificant extra-solution activity (i.e., pre-solution activity) that does not impose meaningfully limits on the claim. (MPEP 2106.04(d)(I), which also refers to MPEP 2106.05(g)).
The claims do not include additional elements/steps that are sufficient to amount to significantly more than the judicial exception. As explained above, these claim limitations recite insignificant extra-solution activity (i.e., pre-solution activity) that does not impose meaningfully limits on the claim. (MPEP 2106.04(d)(I), which also refers to MPEP 2106.05(g)).
Accordingly, claim 12 does not include patent-eligible subject matter.
RESPONSE TO APPLICANT’S ARGUMENTS
Applicant amended the claims to recite that the method is performed by a computer and that the various steps of the computer-implemented method are performed by an electronic data processing device of the computer. Applicant argues that the amendments clarify the method such that it recites more than a mathematical concept and/or mental process. These arguments are not persuasive.
First, the abstract idea necessarily includes a mathematical concept. The step of determining grid data based on the first feature-set data involves identifying locations of the features using geometric calculations. Moreover, tracking the tracking markers includes performing various 3D calculations to determine the position of the marker.
Second, with respect to mental processes being implemented by computers, courts do not “distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer. As the Federal Circuit has explained, ‘[c]ourts have examined claims that required the use of a computer and still found that the underlying, patent-ineligible invention could be performed via pen and paper or in a person’s mind.’ Versata Dev. Group v. SAP Am., Inc., 793 F.3d 1306, 1335, 115 USPQ2d 1681, 1702 (Fed. Cir. 2015). See also Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1318, 120 USPQ2d 1353, 1360 (Fed. Cir. 2016) (‘[W]ith the exception of generic computer-implemented steps, there is nothing in the claims themselves that foreclose them from being performed by a human, mentally or with pen and paper.’); Mortgage Grader, Inc. v. First Choice Loan Servs. Inc., 811 F.3d 1314, 1324, 117 USPQ2d 1693, 1699 (Fed. Cir. 2016) (holding that computer-implemented method for ‘anonymous loan shopping’ was an abstract idea because it could be ‘performed by humans without a computer’).” (MPEP 2106.04(a)(2), III) (see also Part C of MPEP 2106.04(a)(2): “Claims can recite a mental process even if they are claimed as being performed on a computer.”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-8, 10, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Wang, Ben. “LFTag: A scalable visual fiducial system with low spatial frequency.” 2020 2nd International Conference on Advances in Computer Technology, Information Science and Communications (CTISC). IEEE, 2020 (hereinafter “WANG”) (previously cited in the Office Action dated December 3, 2025) and Claus et al. “Reliable automatic calibration of a marker-based position tracking system.” 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05)-Volume 1. Vol. 1. IEEE, 2005 (hereinafter “CLAUS”) and U.S. Patent Appl. Publ. No. 2020/0163739 A1 (hereinafter “MESSINGER”).
With respect to claim 1, WANG teaches a tracking marker set comprising a plurality of tracking markers, each of the tracking markers being configured for being positionally detected and tracked during a procedure. WANG teaches a scalable visual fiducial system that can be used with augmented reality and virtual reality systems. (Title and Abstract). Specifically, WANG teaches “LFTag” which is “a visual fiducial system based on topological detection and relative position data encoding which optimizes data density within spatial frequency constraints. The marker is constructed to resolve rotational ambiguity, which combined with the robust geometric and topological false positive rejection, allows all marker bits to be used for data.” (Abstract). Notably, WANG begins by defining “[v]isual fiducual systems” as consisting of “a set of markers located in the environment which can be uniquely identified and localized by an algorithm.” (Page. 1, left column, first paragraph of Introduction). Comparing LFTag to existing state-of-the-art square binary markers, WANG notes that LFTag
PNG
media_image1.png
476
737
media_image1.png
Greyscale
[AltContent: textbox (Fig. 1 of WANG)]
PNG
media_image2.png
267
259
media_image2.png
Greyscale
[AltContent: textbox (Fig. 3(a) of WANG)][AltContent: textbox (“dimension of the grid pattern”)][AltContent: textbox (“grid spacing of the grid pattern”)][AltContent: textbox (“corner node”)][AltContent: textbox (“side node”)]has “significant advantages in dictionary size and range.” (Abstract). Each of the tracking markers comprising:
a substantially planar substructure having a front face and defining a plane. See Fig. 1 of WANG shown here.
a plurality of features disposed at and optically distinct from the front face. See Figs. 1 and 3(a) of WANG shown here.
wherein the plurality of features comprise:
a first set of features defining, within the plane, a dimension and an orientation of a square grid pattern having an equidistant grid spacing and including nine nodes (See Figs. 1 and 3(a)). The most discussed configuration in WANG is 3x3, which forms a square-like grid pattern having nine nodes.), and the grid spacing of the grid pattern being defined by a distance between a feature disposed at a corner node and a feature disposed at a side node (See annotated Fig. 3(a));
a second set of features being aligned with the grid pattern and defining an optical appearance (NOTE: Similar to “feature 5” in Applicant’s disclosure, middle side features of WANG can be offset a predetermined distance to distinguish a marker from other markers. See Figs. 1 and 3(a) of WANG and page 2, right column, middle paragraph: “The remaining regions are called ‘data’ regions, and each encode two bits in its relative location. Each data region has 4 possible locations, with the region centroids either shifting up or down, and left or right. The structure is demonstrated graphically in figure 5.”).
WANG further teaches that each tracking marker from the plurality of tracking markers differs from each other from the plurality of tracking markers in the optical appearance which is specific for the respective tracking marker. One explicit purpose of WANG is to provide unique markers that can be distinguished from one another. “When compared to existing state-of-the-art square binary markers (AprilTag) and topological markers (TopoTag) in simulation, the proposed fiducial system (LFTag) offers significant advances in dictionary size and range.” (Abstract).
WANG further teaches that the optical appearance is defined by one or more of:
sizes of the plurality of features disposed at associated nodes of the grid pattern, which differ for individual features, wherein the sizes are selected from a limited number of predetermined sizes; and/or
spatial positions of the individual features relative to associated nodes of the grid pattern, wherein the spatial positions are selected from a limited set of predefined spatial positions, wherein distances for features being spaced from their associated node are the same for each feature of the tracking marker. See Figs. 1 and 3(a) in which two corner features are larger than other data features. “Two of the regions (also called “baseline” regions) are larger than the others, and are at opposite sides of the top row of the marker. These points are used to resolve rotational ambiguity… The remaining regions are called ‘data’ regions, and each encode two bits in its relative location. Each data region has 4 possible locations, with the region centroids either shifting up or down, and left or right. The structure is demonstrated graphically in figure 5.” (page 2, right column, middle paragraph). NOTE: Claim 1 recites that the optical appearance is defined by one or more of “sizes…and/or spatial positions” of the features. WANG teaches both as noted above.
WANG does not explicitly teach that the dimension of the grid pattern is defined by four features defining respective corner nodes of the grid pattern. Instead, WANG teaches that the dimension of the grid pattern is defined by two features defining respective corner nodes of the grid pattern (see, e.g., larger “baseline regions”). Moreover, WANG strongly suggests an embodiment in which four features define corner nodes. More specifically, WANG describes “key points” as being the two baseline regions and two data regions. “After the candidate marker regions are found, the next step is to identify the four key points located in the corner [sic] of the marker, two of them being baseline regions and two of them being data regions in the bottom left and bottom right extent of the marker.” (page 3, right column, D. Key Point Identification, first paragraph) (See also Figure 6(e) entitled “Key point identification.”). In the section entitled “Future Work,” WANG explicitly suggests improving performance by fixing the two data key points. In other words, WANG suggests making all four corners fixed. “One of the slower stages of marker detection is initial pose estimation, as 16 candidate poses need to be solved for each marker. This can possibly be improved with a small change to the system so one or both of the data carrying key points stay fixed. If one data carrying key point is fixed, a 4x improvement is expected in both the speed of initial pose estimation, and also in false positive rates. This increases to 16x if both data carrying key points are fixed.” (Page 7, right column, VII. Future Work, second paragraph). This strongly suggests that the four corner “key points” would form a
PNG
media_image5.png
200
400
media_image5.png
Greyscale
square grid pattern in which the dimension of the grid pattern is defined by four features defining respective corner nodes of the grid pattern.
Nonetheless, WANG does not explicitly teach that the four features would define the four corner nodes of a square grid pattern. In the same field of endeavor, CLAUS teaches that “[a]ccurate outdoor and indoor tracking remains a key requirement for augmented reality, and despite significant research effort in markerless approaches [16, 6], fiducial based tracking is still the technology of choice for construction of AR systems which must provide repeatable and reliable performance.” (page 1, Introduction, first paragraph). In CLAUS’s system, each “fiducial” includes four discs. (See Figure 2 of CLAUS shown here). CLAUS’s system receives an input image (Fig. 2(a)) and identifies the four corners (Figure 2(e)). “A given marker will have four dots arranged in a square, with area in the centre of the dot for barcodes, icons, or other identification aids.” (page 3, left column. 4. Fiducial Detection, second paragraph). The four discs or dots are provided for pose estimation. (see, e.g., page 2, right column, paragraph beginning with “At run-time, position estimation follows…” and the subsequently described steps). “As four points is the minimum requirement for a unique solution, only one fiducial need be detected to obtain a pose.” (Step 2).
It would have been obvious to one having ordinary skill in the art at the time of filing to modify the LFTag of WANG such that the four key points of WANG define four corner nodes and a dimension of the grid pattern, as suggested in WANG and taught in CLAUS. WANG describe initially estimating a pose of the marker and suggests that this process would be faster with four the four key points fixed. CLAUS describes a system in which four features define corners of the marker and can be used to estimate the pose of the marker prior to identifying any information from the marker. There would have been a reasonable expectation of success as WANG suggests and CLAUS teaches that a marker can have four features that define the corner nodes.
However, WANG does not explicitly teach at least one feature being aligned with the grid pattern and being disposed offset from the plane.
In the same field of endeavor, MESSINGER relates generally to a fiducial marker, and specifically to a marker that can be tracked optically, such as for image guided surgery. ([0001] and [0002]). Specifically, MESSINGER teaches a “positioning marker.” In a particular embodiment (e.g., Figures 3A and 3B), “[t]he retroreflectors are typically configured to be in a given plane, and in order to enhance the tracking provided by the marker, the marker may comprise a further retroreflector located in a plane different from the given plane.” (See also [0076]: “[A]dditional opening 130 and additional retroreflective surface 134 are configured to form a section 138 of the additional retroreflective surface that is in a different plane from the plane of sections of the retroreflective upper surface formed by openings 106….”).
PNG
media_image6.png
559
873
media_image6.png
Greyscale
It would have been obvious to one having ordinary skill in the art to modify the LFTag of WANG to include at least one feature that is aligned with the grid pattern and is disposed offset from the plane. More specifically, one skilled in the art would position at least one feature of the WANG marker on a different plane to enhance the tracking provided by the marker as taught in MESSINGER. There would have been a reasonable expectation of success as both MESSINGER and WANG teach that markers with features having offset or shifted positions can still be tracked and identified.
With respect to claim 3, WANG does not explicitly teach that the at least one feature offset from the plane is disposed at a central node in between at least two side nodes.
However, MESSINGER teaches that the offset retroreflector may be positioned at a center of the marker and either at a depth within the marker (Figures 3A or 3B) or a height with respect to the face of the marker (Figures 4A-4C).
It would have been obvious to one having ordinary skill in the art to modify the LFTag of WANG to be similar to the MESSINGER mark such that the feature that is offset from the plane is disposed at a central node between at least two side nodes. More specifically, one skilled in the art would position the central feature of the WANG marker (i.e., between two side nodes) on a different plane to enhance the tracking provided by the marker as taught in MESSINGER. There would have been a reasonable expectation of success as both MESSINGER and WANG teach that markers with features having offset or shifted positions can still be tracked and identified.
With respect to claims 4 and 5, WANG teaches that wherein each of the features have one of two predefined sizes (claim 4). See Annotated Figure 4A of WANG in which the feature size can either be a larger border region or a smaller data region. “Two of the regions (also called “baseline” regions) are larger than the others, and are at opposite sides of the top row of the marker. These points are used to resolve rotational ambiguity. The remaining regions are called “data” regions, and each encode two bits in its relative location.” (p.141, III. LFTag Design, bottom left column to top right column).
It would have been obvious to one having ordinary skill in the art to configure the tracking marker to have features of one or two predefined sizes. One would be motivated to have at least two baseline features of a larger size, in order to determine the rotational position, and at least data regions of a smaller size in order to permit enough space to have different positions with respect to the node. There would have been a reasonable expectation of success as WANG teaches that features of different sizes can be used.
WANG also teaches that wherein the optical appearance is defined by the size of features of the first set of features as well as by the size of features of the second set of features (claim 5). The optical appearance would necessarily be based on the different sizes of the features.
With respect to claim 6, WANG does not explicitly teach that all of the features are of the same size. However, with all four “key points” being modified to have fixed positions, discussed above, there would no longer be a reason to differentiate the “baseline regions” from the data regions. Moreover, CLAUS demonstrates that markers having four corner points of the same size and shape can be used to estimate the pose of the marker. MESSINGER demonstrates the same. (See Figures 3A or 3B and Figures 4A-4C).
It would have been obvious to one having ordinary skill in the art to configure the tracking marker such that all of the features are of the same size. One would be motivated to use markers of the same size at each of the four corners to simplify pose estimation (i.e., a feature that is further away from the camera would appear smaller while the same sized feature that is closer to the camera would appear larger) and/or to reduce the cost of manufacturing. There would have been a reasonable expectation of success as WANG teaches that features of a similar size can be used.
With respect to claim 7, the combined teachings of MESSINGER and WANG, as discussed above, teaches that the predefined spatial positions being positions on one of the grid-lines adjacent to the node and/or positions in one of the grid-fields adjacent to the node, wherein the predefined spatial positions comprise a predefined distance from the associated node. More specifically, WANG teaches that “[e]ach data region has 4 possible locations, with the region centroids either shifting up or down, and left or right.” (p.141, III. LFTag Design, bottom left column to top right column). See Annotated Figure 4a above. Examiner is interpreting these shifted positions as the centers being within the grid-fields adjacent to the nodes.
With respect to claim 8, WANG does not explicitly teach the features being disk-shaped and are configured to reflect incident light, wherein the features exhibit a high optical contrast towards the front face of the substructure, wherein the features include a retro-reflective coating. However, each of CLAUS and MESSINGER teach that features may be disk-shaped. For example, CLAUS teaches “[i]n our system, each fiducial comprises four discs….” (page 2, right column, Step 2).
Moreover, MESSINGER also teaches disc-shaped features. (See, e.g., Figure 3A of MESSINGER in which the holes 102 forming the retroreflectors are circular holes), wherein the features exhibit a high optical contrast towards the front face of the substructure ([0056]: “at least some retroreflected radiation” is enough to track the marker) , wherein the features include a retro-reflective coating. ([0061]: Retroreflectors may be made from “retroreflective paint.”)
It would have been obvious to one having ordinary skill in the art at the time of filing to modify the LFTag of WANG to include disk-shaped features as recited in claim 8. One of ordinary skill in the art would have been motivated to provide the disc-shaped features of MESSINGER that have a high optical contrast from a retro-reflective coating in order to provide a more easily detectable feature. There would have been a reasonable expectation of success as MESSINGER teaches such disc-shaped features can be detected by imaging.
With respect to claim 10, WANG does not explicitly teach that the at least one feature offset from the plane is disposed in an indentation or on a projection of the front face of the substructure.
However, MESSINGER teaches that the offset retroreflector may be positioned at a center of the marker and either at a depth (i.e., indentation) within the marker (Figures 3A or 3B) or at a height (i.e., on a projection) with respect to the face of the marker (Figures 4A-4C).
It would have been obvious to one having ordinary skill in the art to modify the LFTag of WANG to be similar to the MESSINGER mark such that the at least one feature offset from the plane is disposed in an indentation or on a projection of the front face of the substructure. More specifically, one skilled in the art would position the central feature of the WANG marker (i.e., between two side nodes) on a different plane to enhance the tracking provided by the marker as taught in MESSINGER. There would have been a reasonable expectation of success as both MESSINGER and WANG teach that markers with features having offset or shifted positions can still be tracked and identified.
With respect to claim 14, WANG teaches a tracking system comprising: at least one tracking marker for being positionally detected and tracked during a procedure via one or more optical cameras. WANG teaches a scalable visual fiducial system that can be used with augmented reality and virtual reality systems. (Title and Abstract). Specifically, WANG teaches “LFTag” which is “a visual fiducial system based on topological detection and relative position data encoding which optimizes data density within spatial frequency constraints. The marker is constructed to resolve rotational ambiguity, which combined with the robust geometric and topological false positive rejection, allows all marker bits to be used for data.” (Abstract). Notably, WANG begins by defining “[v]isual fiducual systems” as consisting of “a set of markers located in the environment which can be uniquely identified and localized by an algorithm.” (Page. 1, left column, first paragraph of Introduction). Comparing LFTag to existing state-of-the-art square binary markers, WANG notes that LFTag has “significant advantages in dictionary size and range.” (Abstract). The at least one tracking marker comprising:
a substantially planar substructure having a front face and defining a plane. See Fig. 1 of WANG shown here.
a plurality of features disposed at and optically distinct from the front face. See Figs. 1 and 3(a) of WANG shown here.
wherein the plurality of features comprise:
a first set of features defining, within the plane, a dimension and an orientation of a square grid pattern having an equidistant grid spacing and including nine nodes (See Figs. 1 and 3(a)). The most discussed configuration in WANG is 3x3, which forms a square-like grid pattern having nine nodes.), and the grid spacing of the grid pattern being defined by a distance between a feature disposed at a corner node and a feature disposed at a side node (See annotated Fig. 3(a));
a second set of features being aligned with the grid pattern and defining an optical appearance (NOTE: Similar to “feature 5” in Applicant’s disclosure, middle side features of WANG can be offset a predetermined distance to distinguish a marker from other markers. See Figs. 1 and 3(a) of WANG and page 2, right column, middle paragraph: “The remaining regions are called ‘data’ regions, and each encode two bits in its relative location. Each data region has 4 possible locations, with the region centroids either shifting up or down, and left or right. The structure is demonstrated graphically in figure 5.”).
WANG further teaches that the optical appearance being defined by one or more of:
sizes of the plurality of features disposed at associated nodes of the grid pattern, which differ for individual features, wherein the sizes are selected from a limited number of predetermined sizes; and/or
spatial positions of the individual features relative to associated nodes of the grid pattern, wherein the spatial positions are selected from a limited set of predefined spatial positions, wherein distances for features being spaced from their associated node are the same for each feature of the tracking marker. See Figs. 1 and 3(a) in which two corner features are larger than other data features. “Two of the regions (also called “baseline” regions) are larger than the others, and are at opposite sides of the top row of the marker. These points are used to resolve rotational ambiguity… The remaining regions are called ‘data’ regions, and each encode two bits in its relative location. Each data region has 4 possible locations, with the region centroids either shifting up or down, and left or right. The structure is demonstrated graphically in figure 5.” (page 2, right column, middle paragraph). NOTE: Claim 1 recites that the optical appearance is defined by one or more of “sizes…and/or spatial positions” of the features. WANG teaches both as noted above.
WANG does not explicitly teach that the dimension of the grid pattern is defined by four features defining respective corner nodes of the grid pattern. Instead, WANG teaches that the dimension of the grid pattern is defined by two features defining respective corner nodes of the grid pattern (see, e.g., larger “baseline regions”). Moreover, WANG strongly suggests an embodiment in which four features define corner nodes. More specifically, WANG describes “key points” as being the two baseline regions and two data regions. “After the candidate marker regions are found, the next step is to identify the four key points located in the corner [sic] of the marker, two of them being baseline regions and two of them being data regions in the bottom left and bottom right extent of the marker.” (page 3, right column, D. Key Point Identification, first paragraph) (See also Figure 6(e) entitled “Key point identification.”). In the section entitled “Future Work,” WANG explicitly suggests improving performance by fixing the two data key points. In other words, WANG suggests making all four corners fixed. “One of the slower stages of marker detection is initial pose estimation, as 16 candidate poses need to be solved for each marker. This can possibly be improved with a small change to the system so one or both of the data carrying key points stay fixed. If one data carrying key point is fixed, a 4x improvement is expected in both the speed of initial pose estimation, and also in false positive rates. This increases to 16x if both data carrying key points are fixed.” (Page 7, right column, VII. Future Work, second paragraph). This strongly suggests that the four corner “key points” would form a
PNG
media_image5.png
200
400
media_image5.png
Greyscale
square grid pattern in which the dimension of the grid pattern is defined by four features defining respective corner nodes of the grid pattern.
Nonetheless, WANG does not explicitly teach that the four features would define the four corner nodes of a square grid pattern. In the same field of endeavor, CLAUS teaches that “[a]ccurate outdoor and indoor tracking remains a key requirement for augmented reality, and despite significant research effort in markerless approaches [16, 6], fiducial based tracking is still the technology of choice for construction of AR systems which must provide repeatable and reliable performance.” (page 1, Introduction, first paragraph). In CLAUS’s system, each “fiducial” includes four discs. (See Figure 2 of CLAUS shown here). CLAUS’s system receives an input image (Fig. 2(a)) and identifies the four corners (Figure 2(e)). “A given marker will have four dots arranged in a square, with area in the centre of the dot for barcodes, icons, or other identification aids.” (page 3, left column. 4. Fiducial Detection, second paragraph). The four discs or dots are provided for pose estimation. (see, e.g., page 2, right column, paragraph beginning with “At run-time, position estimation follows…” and the subsequently described steps). “As four points is the minimum requirement for a unique solution, only one fiducial need be detected to obtain a pose.” (Step 2).
It would have been obvious to one having ordinary skill in the art at the time of filing to modify the LFTag of WANG such that the four key points of WANG define four corner nodes and a dimension of the grid pattern, as suggested in WANG and taught in CLAUS. WANG describe initially estimating a pose of the marker and suggests that this process would be faster with four the four key points fixed. CLAUS describes a system in which four features define corners of the marker and can be used to estimate the pose of the marker prior to identifying any information from the marker. There would have been a reasonable expectation of success as WANG suggests and CLAUS teaches that a marker can have four features that define the corner nodes.
However, WANG does not explicitly teach at least one feature being aligned with the grid pattern and being disposed offset from the plane.
PNG
media_image6.png
559
873
media_image6.png
Greyscale
In the same field of endeavor, MESSINGER relates generally to a fiducial marker, and specifically to a marker that can be tracked optically, such as for image guided surgery. ([0001] and [0002]). Specifically, MESSINGER teaches a “positioning marker.” In a particular embodiment (e.g., Figures 3A and 3B), “[t]he retroreflectors are typically configured to be in a given plane, and in order to enhance the tracking provided by the marker, the marker may comprise a further retroreflector located in a plane different from the given plane.” (See also [0076]: “[A]dditional opening 130 and additional retroreflective surface 134 are configured to form a section 138 of the additional retroreflective surface that is in a different plane from the plane of sections of the retroreflective upper surface formed by openings 106….”).
It would have been obvious to one having ordinary skill in the art to modify the LFTag of WANG to include at least one feature that is aligned with the grid pattern and is disposed offset from the plane. More specifically, one skilled in the art would position at least one feature of the WANG marker on a different plane to enhance the tracking provided by the marker as taught in MESSINGER. There would have been a reasonable expectation of success as both MESSINGER and WANG teach that markers with features having offset or shifted positions can still be tracked and identified.
WANG also does not explicitly teach that the tracking system is a medical tracking system or that the marker is traced during a procedure or that the system also includes a computer configured to identify and positionally track the at least one tracking marker during a procedure, wherein the computer is operable to distinguish between a plurality of tracking markers based on the optical appearance of the respective tracking markers.
MESSINGER teaches that the markers may be used during image-guided surgery. ([0002]). “During image guide surgery it is typically necessary to track objects used in the surgery, and/or elements of the patient undergoing the surgery.” To this end, MESSINGER teaches using a computer 28 having a processor 26 capable of tracking the markers. (see, e.g., Figure 1). “As is also described below, processor 26 is able to track the location of marker 18, and thus, since the marker is at a predefined location with respect to ROI 34, the processor is able to track the location of the ROI.” ([0046]). “Alternatively or additionally, positioning marker 18, or a marker substantially similar to marker 18, may be attached to a tool or other device used in the procedure. In this case processor 26 is able to track the marker and thus the tool or device attached to the marker.” ([0047]). Thus, MESSINGER teaches a system capable of tracking a patient with a first marker and a tool or other device with a second marker.
It would have been obvious to one having ordinary skill in the art at the time of filing to use a computer to track and identify the different WANG markers (as modified) that are used during a medical procedure, as taught in MESSINGER. One having ordinary skill in the art would have been motivated to use a computer that is operably connected to the cameras that track the patient and the different tools or devices that are used during a medical procedure, such as those medical procedures that use augmented reality as taught in MESSINGER. There would have been a reasonable expectation of success as MESSINGER teaches that markers can be tracked during medical procedures.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Wang, Ben. “LFTag: A scalable visual fiducial system with low spatial frequency.” 2020 2nd International Conference on Advances in Computer Technology, Information Science and Communications (CTISC). IEEE, 2020 (hereinafter “WANG”) and Claus et al. “Reliable automatic calibration of a marker-based position tracking system.” 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05)-Volume 1. Vol. 1. IEEE, 2005 (hereinafter “CLAUS”) and U.S. Patent Appl. Publ. No. 2020/0163739 A1 (hereinafter “MESSINGER”) as applied to claim 1 above, and further in view of U.S. Patent Appl. Publ. No. 2017/0086941 A1 (hereinafter “MARTI”) and Wang XY, Liu L, Guan MS, Liu Q, Zhao T, Li HB. The accuracy and learning curve of active and passive dynamic navigation-guided dental implant surgery: An in vitro study. Journal of Dentistry. 2022 Sep 1;124:104240 (hereinafter XIAO-YU).
With respect to claim 9, neither WANG nor MESSINGER nor CLAUS teach that the each of the plurality of features comprises a light-emitting element.
In the same field of endeavor, MARTI teaches multiple designs for markers of optical tracking systems. (Title and [0003]). MARTI’s teaches similar configurations in which the one is an active configuration and the other is a passive configuration. (Compare Figures 1 and 2 (passive) to Figure 6 (active)). See also [0003]: “Traditional optical pose tracking systems comprise two cameras. They use triangulation to determine the three-dimensional (3D) position of light generating elements in space. These light generating elements may either be active: they transmit light (e.g. LEDs) or passive: they reflect light (e.g. reflective disks or spheres), or a combination of active and passive.”)
While MARTI teaches that active LEDs can be used to replace passive reflectors, XIAO-YU teaches that, for some circumstances such as dental implantation, active dynamic navigation is superior to the passive. “In the present study, the accuracy of active dynamic navigation system was greater than that of passive dynamic navigation system.” (Discussion). “In our study, the accuracy of M-PBR method was significantly better than that of F-PBR method for all the evaluated accuracy indicators (P < 0.01).” (Id). Moreover, the active system was quicker to learn. (Id).
It would have been obvious to one having ordinary skill in the art to substitute the features of WANG with the active LEDs as taught in MARTI. One would have chosen to use active LEDs over dark features because, for certain procedures, active navigation systems perform better as taught in XIAO-YU. There would have been a reasonable expectation of success as MARTI teaches that similar configurations of tracking markers can use active LEDs instead of retroreflectors.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Appl. Publ. No. 2020/0163739 A1 (hereinafter “MESSINGER”) and Wang, Ben. “LFTag: A scalable visual fiducial system with low spatial frequency.” 2020 2nd International Conference on Advances in Computer Technology, Information Science and Communications (CTISC). IEEE, 2020 (hereinafter “WANG”).
With respect to claim 12, MESSINGER teaches a computer-implemented method of identifying and positionally tracking a tracking marker during a procedure. (see [0045] and [0046] describing processor identifying an ROI of a patient using the marker 18).
As discussed above with respect to claim 1, MESSINGER teaches an optical marker that includes first features, second features, and at least one feature offset from the plane. (The features are separate portions of the retroreflective base 80 that are exposed by openings in the cover 90. See, e.g., [0064]: “When cover 90 is fastened to base 80, surface 92 of the cover mates with the retroreflective upper surface of base 80, and smaller circular openings 106 contact base 80, forming retroreflective circles 96, herein termed retroflectors 96.” See also [0066]: “[I]t will be appreciated that the retroreflective upper surface of base 80 is visible, through the openings, from many different directions. This visibility facilitates the ability of processor 26 to track marker 18 using camera 72 if there is relative motion between the camera and the marker, for example if professional 22 moves.”). With respect to the off-plane feature, MESSINGER teaches that “in order to enhance the tracking provided by the marker, the marker may comprise a further retroreflector located in a plane different from the given plane.” ([0040]) (emphasis added). See also [0076]: “[A]dditional opening 130 and additional retroreflective surface 134 are configured to form a section 138 of the additional retroreflective surface that is in a different plane from the plane of sections of the retroreflective upper surface formed by openings 106….”) (emphasis added).
MESSINGER teaches acquiring images of first features, second features, and at least one feature that is off-plane. ([0038]: “Embodiments of the present invention provide a positioning marker which facilitates the registration referred to above, by enabling a processor coupled to the augmented reality assembly to track the marker, and to maintain the tracking even when large changes of angle, subtended by the marker to the assembly, occur.” See also [0081]: “By forming opening 130 as a frustum, retroreflector 138 [i.e., corresponding to off-plane data] is visible (as with retroflectors 96 of openings 98 [i.e., corresponding to first and second feature-set data]) from many different directions, so facilitating the ability of processor 26 to track marker 18 using camera 72.”). MESSINGER also teaches acquiring off-plane data that describes a position of at least one feature offset from the plane, with respect to the first set of features and/or with respect to the second set of features and determining tracking data based on at least one of the first feature-set data, the second feature-set data and the off-plane data, wherein the tracking data describes a spatial position of the identified tracking marker. (See [0074]: “To further facilitate tracking of marker 18, the marker comprises an additional opening 130 in cover 90, and an associated additional retroreflective surface 134 formed on an additional sheet 136.” See also [0046]: “In embodiments of the present invention professional 22 enables processor 26 to identify the ROI by locating positioning marker 18, described in more detail below, at a predefined location with respect to ROI 34, e.g., the ROI may be a predefined distance to the right and a predefined distance below marker 18. As is also described below, processor 26 is able to track the location of marker 18, and thus, since the marker is at a predefined location with respect to ROI 34, the processor is able to track the location of the ROI.” See also [0047] teaching that the positioning marker may be attached to a tool to track the tool).
MESSINGER teaches that the markers may be used during image-guided surgery. ([0002]). “During image guide surgery it is typically necessary to track objects used in the surgery, and/or elements of the patient undergoing the surgery.” To this end, MESSINGER performs the various steps using an electronic data processing device. More specifically, MESSINGER teaches using a computer 28 having a processor 26 capable of tracking the markers. (see, e.g., Figure 1). “Processor 26 is typically able to access a database 40, wherein are stored images and other visual elements used by system 20.” ([0045]). “As is also described below, processor 26 is able to track the location of marker 18, and thus, since the marker is at a predefined location with respect to ROI 34, the processor is able to track the location of the ROI.” ([0046]). “Alternatively or additionally, positioning marker 18, or a marker substantially similar to marker 18, may be attached to a tool or other device used in the procedure. In this case processor 26 is able to track the marker and thus the tool or device attached to the marker.” ([0047]). Thus, MESSINGER teaches a system capable of tracking a patient with a first marker and a tool or other device with a second marker.
However, MESSINGER does not explicitly teach the remaining limitations of claim 12. In the same field of endeavor, WANG teaches:
acquiring first feature-set data that describes a position of a first set of features of the tracking marker within a plane of an image obtained via an optical camera. After filtering the image data to determine candidate marker regions, WANG first assigns the largest regions as the baseline regions. See “Key Point Identification” section at paragraph beginning with “[f]irst, the regions are sorted by the inverted zeroth order image moment, after dilation by δ pixels. The two largest regions are picked as the baseline regions, denoted in arbitrary order by Ba and Bb.” (p.142, D. Key Point Identification, right column, second paragraph).
determining grid data based on the first feature-set data, wherein the grid data describes a position of a grid pattern of the tracking marker within the plane of the image. Knowing the two larger regions are the baseline regions, WANG then determines which of the two baseline regions each is. See “Key Point Identification” section at paragraph beginning with “[i]n order to disambiguate the order of these two regions, first a vector is found from Ba to Bb. Then, for every region in the marker, a vector is also calculated from Ba….” (p.142, D. Key Point Identification, right column, fourth paragraph).
acquiring second features-set data that describes a position of a second set of features of the tracking marker within the plane of the image, particularly with respect to the first set of features. After determining two corners of the baseline regions (called “key points”), WANG teaches finding the other two key points (i.e., corners) that are formed by the smaller, data regions. See “Key Point Identification” section at paragraph beginning with “[t]o locate the remaining two key points, a vector is drawn from K0 to K1, and for each remaining region, a vector is found from K0….” (p.142, D. Key Point Identification, right column, last paragraph). After determining the key points, WANG teaches confirming that all data points (other than the row with the baseline regions) are on the same side of the row with the baseline regions. “All data points are checked to ensure that all but n−2 data region centroids lie on the same side of the baseline vector. (This is also called the geometry constraint.)” (p.143, E. Geometry Filtering, left column, first paragraph).
determining identification data based on the first feature-set data and the second feature-set data, wherein the identification data describes an identity of the tracking marker. After determining a pose estimation based on the four key points (corners), WANG decodes the information from baseline and data regions. See “Tag Decoding” section beginning with “For each estimated pose, a decoding is also attempted….” (p.143, H. Tag Decoding, left column).
It would have also been obvious to one having ordinary skill in the art to use the image processing steps of WANG and analyze the first-features set data to identify the relative locations of the fixed corner features of MESSINGER and then use that information to identify the locations of the features in the second features-set data relative to the fixed corner features, as taught in WANG. Knowing the locations of the first and second features, one skilled in the art would then determining the identification data (i.e., decode) of the tracking marker as taught in WANG. One would be motivated to use this process because WANG’s use of “topological filtering produces roughly an order of magnitude less candidate regions compared to state of the art square marker detectors…,” thereby reducing workload and computation time (p.146, Discussion, left column). There would have been a reasonable expectation of success as WANG teaches the process may be used to determine position and identification data.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Appl. Publ. No. 2020/0163739 A1 (hereinafter “MESSINGER”) and Wang, Ben. “LFTag: A scalable visual fiducial system with low spatial frequency.” 2020 2nd International Conference on Advances in Computer Technology, Information Science and Communications (CTISC). IEEE, 2020 (hereinafter “WANG”) as applied to claim 12 above, and further in view of Kalaitzakis, Michail, et al. “Fiducial markers for pose estimation: Overview, applications and experimental comparison of the artag, apriltag, aruco and stag markers.” Journal of Intelligent & Robotic Systems 101.4 (2021): 71. (hereinafter “KALAITZAKIS”).
With respect to claim 13, MESSINGER does not teach using, by the electronic data processing device of the computer, a monochrome mono-camera, that is configured to optically detect and distinguish the plurality of features from front face of the substructure. However, MESSINGER does teach using infra-red (single channel). Moreover, WANG teaches that the fiducial system can be used for “monocular pose estimation.” (Abstract and first paragraph).
In the same field of endeavor, KALAITZAKIS shows that overwhelming majority of fiducial marker system use topological encoding (like MESSINGER and WANG) and are monochromatic. (See, e.g., Table 1).
It would have been obvious to one having ordinary skill in the art to use a monochrome mono-camera as taught in WANG and KALAITZAKIS that is configured to optically detect and distinguish the plurality of features from front face of the substructure. One would be motivated to use the monochrome mono-camera systems as it has been proven to be successful in tracking planar markers. There would have been a reasonable expectation of success as WANG and KALAITZAKIS demonstrate that monochrome mono-camera systems can be used effectively to track planar markers.
RESPONSE TO APPLICANT’S ARGUMENTS
Applicant's arguments filed April 3, 2026 have been fully considered but they are not persuasive.
With respect to claims 1 and 14, Applicant argues that the MESSINGER reference fails to teach a grid pattern having equi-distant spacing. (see page 12 of Response). As discussed above, however, Examiner is relying upon WANG for teaching this feature.
With respect to claims 1 and 14, Applicant also argues that WANG does not teach the dimensions of the grid pattern being defined by the four corner features. As discussed above, Examiner relies upon a newly cited reference, CLAUS, for teaching this feature.
Applicant states that claim 12 was amended in a similar manner to claim 1. This is not correct. Claim 12 was only amended to recite that the various steps of the method were performed by an electronic data processing device of the computer. As discussed above MESSINGER teaches this claim limitation.
Prior Art Made of Record
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US-20220175464-A1 describes trackers having four corners or “blobs” that are used during pose estimation. (See, e.g., Figures 4, 5, 7, and 8).
US-20220323175-A1 describes identifying a pose of a square-shaped marker first and then identifying the particular marker. (See, e.g., Figures 15D-15I and accompanying description).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASON P GROSS whose telephone number is (571)272-1386. The examiner can normally be reached Monday-Friday 9:00-5:00CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne M. Kozak can be reached at (571) 270-5284. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JASON P GROSS/ Examiner, Art Unit 3797
/SERKAN AKAR/ Primary Examiner, Art Unit 3797
Prosecution Insights